Linux: BtrFS
-
@stacksofplates said in Linux: BtrFS:
@scottalanmiller said in Linux: BtrFS:
@msff-amman-Itofficer said in Linux: BtrFS:
... imagine I will run KVM server ontop of BTRFS, will it makes sense to snapshot the volume ?
Did it make sense before with LVM and XFS? No, and the reason is because you can't snap outside of the VM because there isn't enough information to know if the data is quiesced. Snapping under the hood at the LVM2 or BtrFS layer would be the same as snapping on a SAN which, of course, can cause corruption. So we avoid that. We need the VM itself to have this done via the hypervisor.
I've specifically seen you recommend using LVM as a backup mechanism with KVM.
Have I recommended it, or have I recommended it within the context of taking image based backups?
-
@scottalanmiller said in Linux: BtrFS:
@stacksofplates said in Linux: BtrFS:
@scottalanmiller said in Linux: BtrFS:
@msff-amman-Itofficer said in Linux: BtrFS:
... imagine I will run KVM server ontop of BTRFS, will it makes sense to snapshot the volume ?
Did it make sense before with LVM and XFS? No, and the reason is because you can't snap outside of the VM because there isn't enough information to know if the data is quiesced. Snapping under the hood at the LVM2 or BtrFS layer would be the same as snapping on a SAN which, of course, can cause corruption. So we avoid that. We need the VM itself to have this done via the hypervisor.
I've specifically seen you recommend using LVM as a backup mechanism with KVM.
Have I recommended it, or have I recommended it within the context of taking image based backups?
That's what he was talking about above. Using BtrFS to take a snapshot of the volume, like using LVM to take a snapshot of the volume.
-
@stacksofplates said in Linux: BtrFS:
@scottalanmiller said in Linux: BtrFS:
@stacksofplates said in Linux: BtrFS:
@scottalanmiller said in Linux: BtrFS:
@msff-amman-Itofficer said in Linux: BtrFS:
... imagine I will run KVM server ontop of BTRFS, will it makes sense to snapshot the volume ?
Did it make sense before with LVM and XFS? No, and the reason is because you can't snap outside of the VM because there isn't enough information to know if the data is quiesced. Snapping under the hood at the LVM2 or BtrFS layer would be the same as snapping on a SAN which, of course, can cause corruption. So we avoid that. We need the VM itself to have this done via the hypervisor.
I've specifically seen you recommend using LVM as a backup mechanism with KVM.
Have I recommended it, or have I recommended it within the context of taking image based backups?
That's what he was talking about above. Using BtrFS to take a snapshot of the volume, like using LVM to take a snapshot of the volume.
Well yes, but what I mean is if someone asks "how do you take backups" I normally recommend an agent. If someone says "how do I snap images" I normally say LVM.
The difference is if the goal is to backup the system or if the question is "can KVM do image backups."
-
@scottalanmiller said in Linux: BtrFS:
@BBigford said in Linux: BtrFS:
If it's "Better File System", why not pronounce it better, rather than butter?
I've never understood that either, and a few people do, but somehow "butter" became the de facto standard, even before it was available for general use. My only guess is that they did not want to come off as pretentious.
Lol then pronounce it 'butter' and call it 'Butter File System'?
-
@BBigford said in Linux: BtrFS:
@scottalanmiller said in Linux: BtrFS:
@BBigford said in Linux: BtrFS:
If it's "Better File System", why not pronounce it better, rather than butter?
I've never understood that either, and a few people do, but somehow "butter" became the de facto standard, even before it was available for general use. My only guess is that they did not want to come off as pretentious.
Lol then pronounce it 'butter' and call it 'Butter File System'?
It is what it is, I just report the news
-
@scottalanmiller said in Linux: BtrFS:
@stacksofplates said in Linux: BtrFS:
@scottalanmiller said in Linux: BtrFS:
@stacksofplates said in Linux: BtrFS:
@scottalanmiller said in Linux: BtrFS:
@msff-amman-Itofficer said in Linux: BtrFS:
... imagine I will run KVM server ontop of BTRFS, will it makes sense to snapshot the volume ?
Did it make sense before with LVM and XFS? No, and the reason is because you can't snap outside of the VM because there isn't enough information to know if the data is quiesced. Snapping under the hood at the LVM2 or BtrFS layer would be the same as snapping on a SAN which, of course, can cause corruption. So we avoid that. We need the VM itself to have this done via the hypervisor.
I've specifically seen you recommend using LVM as a backup mechanism with KVM.
Have I recommended it, or have I recommended it within the context of taking image based backups?
That's what he was talking about above. Using BtrFS to take a snapshot of the volume, like using LVM to take a snapshot of the volume.
Well yes, but what I mean is if someone asks "how do you take backups" I normally recommend an agent. If someone says "how do I snap images" I normally say LVM.
The difference is if the goal is to backup the system or if the question is "can KVM do image backups."
I don't understand what this has to do with anything. Above you said
Did it make sense before with LVM and XFS? No, and the reason is because you can't snap outside of the VM because there isn't enough information to know if the data is quiesced.
It does make sense. You can easily suspend the VM and take a LV snapshot.
You've also said this to someone else which is what I was referring to:
Why are you not using logical volumes? that is both a general Linux best practice as well as the backup method for KVM.
Then you linked a python script that does exactly what I said. Suspends the VM and takes a snapshot of the logical volume.
My point is, why are you saying here that it makes no sense?
-
@stacksofplates said in Linux: BtrFS:
Did it make sense before with LVM and XFS? No, and the reason is because you can't snap outside of the VM because there isn't enough information to know if the data is quiesced.
It does make sense. You can easily suspend the VM and take a LV snapshot.
You've also said this to someone else which is what I was referring to:
Why are you not using logical volumes? that is both a general Linux best practice as well as the backup method for KVM.
Then you linked a python script that does exactly what I said. Suspends the VM and takes a snapshot of the logical volume.
My point is, why are you saying here that it makes no sense?
Does it? Who really feels that a "you have to shut down the host" backup method is generally valid? I think most companies would consider that a bit of a fail. Does it work? Sure. Does it make sense? Not really. Once you are shutting down the VM, just copy the files, no need to snap the whole underlying volume. Are there benefits to the snap? A little, but you are getting into a less than ideal zone here. I'm certainly not pushing that.
So did it make sense before, no. Does it change with BtrFS, no. Are there edge cases where you can make it work or it might make sense in niche cases? Yes, but not in general ones.
To someone else I was talking about the importance of having LVM and taking a snap inside Linux is quite different than taking one under it. And it is the backup method of KVM, which many people want. That it is there and that I recommend it are not related.
I feel that everything that you quoted from me is consistent. Different use cases. You have to read into it something that I didn't say to have a conflict. KVM has LVM as its snap / image backup method. I rarely would recommend using that. I would never consider running a production system without snap capacity, though. It is a best practice to keep LVM in Linux. Using snaps without quiescence as a backup system is not ideal regardless of if it is LVM or BtrFS, yes you can force quintessence by offlining the system, but I've not recommended that either, not outside of an emergency measure.
-
I think the gap here might be the difference between recommending having a capacity to use a mechanism versus recommending using that mechanism in a given manner.
-
@scottalanmiller said in Linux: BtrFS:
@stacksofplates said in Linux: BtrFS:
Did it make sense before with LVM and XFS? No, and the reason is because you can't snap outside of the VM because there isn't enough information to know if the data is quiesced.
It does make sense. You can easily suspend the VM and take a LV snapshot.
You've also said this to someone else which is what I was referring to:
Why are you not using logical volumes? that is both a general Linux best practice as well as the backup method for KVM.
Then you linked a python script that does exactly what I said. Suspends the VM and takes a snapshot of the logical volume.
My point is, why are you saying here that it makes no sense?
Does it? Who really feels that a "you have to shut down the host" backup method is generally valid? I think most companies would consider that a bit of a fail. Does it work? Sure. Does it make sense? Not really. Once you are shutting down the VM, just copy the files, no need to snap the whole underlying volume. Are there benefits to the snap? A little, but you are getting into a less than ideal zone here. I'm certainly not pushing that.
So did it make sense before, no. Does it change with BtrFS, no. Are there edge cases where you can make it work or it might make sense in niche cases? Yes, but not in general ones.
To someone else I was talking about the importance of having LVM and taking a snap inside Linux is quite different than taking one under it. And it is the backup method of KVM, which many people want. That it is there and that I recommend it are not related.
I feel that everything that you quoted from me is consistent. Different use cases. You have to read into it something that I didn't say to have a conflict. KVM has LVM as its snap / image backup method. I rarely would recommend using that. I would never consider running a production system without snap capacity, though. It is a best practice to keep LVM in Linux. Using snaps without quiescence as a backup system is not ideal regardless of if it is LVM or BtrFS, yes you can force quintessence by offlining the system, but I've not recommended that either, not outside of an emergency measure.
You're not "shutting down the host". It's suspended, completely different. It's perfectly valid because when you take an internal snapshot with a qcow2 file, it does exactly the same thing.
You were not talking about using internal snapshots at all. He was asking how to spin up a backup of a VM and you recommended using logical volume snapshots. Don't try to change what you said. It wasn't an inconsistent quote, you said exactly what I posted in the same manner we are talking about here.
-
@stacksofplates said in Linux: BtrFS:
You were not talking about using internal snapshots at all. He was asking how to spin up a backup of a VM and you recommended using logical volume snapshots. Don't try to change what you said. It wasn't an inconsistent quote, you said exactly what I posted in the same manner we are talking about here.
You only posted my portion of it, what's the whole thing?
-
@scottalanmiller said in Linux: BtrFS:
@stacksofplates said in Linux: BtrFS:
You were not talking about using internal snapshots at all. He was asking how to spin up a backup of a VM and you recommended using logical volume snapshots. Don't try to change what you said. It wasn't an inconsistent quote, you said exactly what I posted in the same manner we are talking about here.
You only posted my portion of it, what's the whole thing?
-
@stacksofplates said in Linux: BtrFS:
Why are you not using logical volumes? that is both a general Linux best practice as well as the backup method for KVM.
Here we go. Clearly I was responding to why it made no sense for him to be removing LVM. It in no way was a recommendation to use this as the backup. You changed the context.
It is the backup method for KVM. But I didn't recommend it, there. Also notice that this is FIVE years old. Half a decade. Saying that I'm inconsistent over half a decade is a bit extreme. I've leaned heavily away from image based backups over half a decade, but even back then when I normally recommended them, I didn't recommend it here.
-
You have to look at the context of the original thread, as well. Just helping someone who was trying to figure out how to do a specific task, in a lab, long ago. He wanted a specific thing. And tools like we have today otherwise were not the same or as mature or free or whatever either.
For example, if this quote was from before good, enterprise agents were available for free or DevOps tools were wisely known or free, the context has to be understood as working with what he had at the time. Veeam agents and DevOps tools, for example, have significantly changed what good looks like in a lot of scenarios.
-
@scottalanmiller said in Linux: BtrFS:
@stacksofplates said in Linux: BtrFS:
Why are you not using logical volumes? that is both a general Linux best practice as well as the backup method for KVM.
Here we go. Clearly I was responding to why it made no sense for him to be removing LVM. It in no way was a recommendation to use this as the backup. You changed the context.
It is the backup method for KVM. But I didn't recommend it, there. Also notice that this is FIVE years old. Half a decade. Saying that I'm inconsistent over half a decade is a bit extreme. I've leaned heavily away from image based backups over half a decade, but even back then when I normally recommended them, I didn't recommend it here.
No, I didn't change the context because the context was never "I want to remove logical volumes". He didn't have it implemented yet. How is saying it is "the backup method for KVM" not recommending it.
This would all be half believable if the very next post you made wasn't a script doing exactly what I described. Suspending the VM and taking a snapshot of the logical volume.
-
@stacksofplates said in Linux: BtrFS:
No, I didn't change the context because the context was never "I want to remove logical volumes". He didn't have it implemented yet. How is saying it is "the backup method for KVM" not recommending it.
Because it simply isn't. The sky is blue does not recommend that you go flying. I'm totally lost how you see me making a factual statement as being a recommendation. The two are unrelated.
I DID recommend that he have LVM, because that's just best practice. But my point was that he had installed Linux without it, which I would never recommend doing.
-
@scottalanmiller said in Linux: BtrFS:
You have to look at the context of the original thread, as well. Just helping someone who was trying to figure out how to do a specific task, in a lab, long ago. He wanted a specific thing. And tools like we have today otherwise were not the same or as mature or free or whatever either.
For example, if this quote was from before good, enterprise agents were available for free or DevOps tools were wisely known or free, the context has to be understood as working with what he had at the time. Veeam agents and DevOps tools, for example, have significantly changed what good looks like in a lot of scenarios.
And you never do this any other time. You dig until you find out why they are doing something incorrectly, and have chastised people for just giving answers. You've also told people that they should be using the image files vs block storage when possible, but never mentioned it here. Again, none of the evidence you are providing points to the fact that you didn't think this was the correct way to do it.
-
@scottalanmiller said in Linux: BtrFS:
@stacksofplates said in Linux: BtrFS:
No, I didn't change the context because the context was never "I want to remove logical volumes". He didn't have it implemented yet. How is saying it is "the backup method for KVM" not recommending it.
Because it simply isn't. The sky is blue does not recommend that you go flying. I'm totally lost how you see me making a factual statement as being a recommendation. The two are unrelated.
I DID recommend that he have LVM, because that's just best practice. But my point was that he had installed Linux without it, which I would never recommend doing.
Ah so you said it's the back up method, but didn't clarify that it's not the recommended method (which we all know you would never leave that out). So again, I don't believe that you didn't fully intend this as a recommendation.
-
Let me give another example: the default filesystem is Suse is BtrFS.
Under no circumstances does that mean that I just recommend that you use BtrFS on Suse. If that were true, things like factsheets and datasheets would be sets of recommendations rather than lists of specs.
If I said "and you should use it" or "and I recommend it" then that would be a recommendation.
The reason that I was so adamant there is because you should not be installing Linux without it. The OP was about how he hadn't installed LVM, I responded primarily to that. The mentioning of it being useful for KVM was not even the focus but an offhand remark.
-
Similarly... Windows has recently introduced ReFS.
Again, just information, not a recommendation. In fact, I'd almost never recommend ReFS. NTFS is generally better.
Of course, half a decade from now, that might change as other factors change around it.
-
@scottalanmiller said in Linux: BtrFS:
Let me give another example: the default filesystem is Suse is BtrFS.
Under no circumstances does that mean that I just recommend that you use BtrFS on Suse. If that were true, things like factsheets and datasheets would be sets of recommendations rather than lists of specs.
If I said "and you should use it" or "and I recommend it" then that would be a recommendation.
The reason that I was so adamant there is because you should not be installing Linux without it. The OP was about how he hadn't installed LVM, I responded primarily to that. The mentioning of it being useful for KVM was not even the focus but an offhand remark.
No it wasn't, omg. The thread is titled "What is the best method for making backups of KVM virtual machines." Come on......
