Someone doesn't like local storage for large amounts of data
-
By the logic that big VDIs defeat the purpose of virtualization because they cannot be moved, that also would undermine the accepted best practice of virtualizing when you have only a single physical server, but as we know you would always do that and the benefits are myriad even without that one benefit.
Things like driver stability, standard environments, snapshotting from outside of the OS, greater flexibility to combat the unknown, etc. are big deals.
And remember, you are only saying that you can't move the VDI as easily, but by going to physical it because "can't move it at all." Doing something in a difficult way is still easier than not being able to do something at all. So virtual still benefits from it, just not as much.
-
@olivier said in Someone doesn't like local storage for large amounts of data:
Okay, let's imagine your NAS is on a larger VMs talking all space of your local storage of your XenServer/ESXi/whatever host. And exposing mounts to all other VMs on other hosts. That's doable but that's not my point. For me, this setup is almost the same of having a physical NAS/SAN.
Sure, ALMOST the same, but still a little better. That's the point. The VM approach still has benefits, the NAS is still a little worse. Why opt for worse when better is free?
-
@scottalanmiller That's not my point. A big virtualized NAS is doable if you like. I was talking about the architecture. Damn, I have the impression to speak Chinese. Sorry if I can't express clearly my ideas in English.
-
@olivier said in Someone doesn't like local storage for large amounts of data:
@scottalanmiller I was talking about the architecture.
Do you mean that if you have massive file serving needs that it makes the most sense to have that on unique hardware that is then shared to other VMs? That makes sense, but seems like a standard capacity algorithm solution rather than a special case. Any workload that becomes dramatically unbalanced from the others would operate in that way.
-
@scottalanmiller Because that's not my point aaarrrgghh. I don't care, that's a not something I wanted to focus in my opinion at the first place.
-
I think that we are all lost at this point. Maybe start over and word it fresh. What @Dashrender had in the original post was not at all what you had said.
I thought that you meant physical file server was better than a virtual one, but that wasnt it either.
I don't know what was originally said that prompted the conversation so only working from what is in the thread.
-
The thing, initially, was about having VMs with large VDIs. Which is for me not a good practice.
But if you need to store a large amount of data, it's better to connect to a remote file share in the VM and keep small system disks (excepts for db/web usage, which are not huge in general).
That's all.
edit: is it more clear now?
-
@olivier said in Someone doesn't like local storage for large amounts of data:
The thing, initially, was about having VMs with large VDIs. Which is for me not a good practice.
But if you need to store a large amount of data, it's better to connect to a remote file share in the VM and keep small system disks (excepts for db/web usage, which are not huge in general).
That's all.
edit: is it more clear now?
Let's see if I reword it correctly....
If your VM needs a lot of file storage.... then it is better to mount that from a file server rather than keeping it in the original VM?
-
@scottalanmiller said in Someone doesn't like local storage for large amounts of data:
@olivier said in Someone doesn't like local storage for large amounts of data:
The thing, initially, was about having VMs with large VDIs. Which is for me not a good practice.
But if you need to store a large amount of data, it's better to connect to a remote file share in the VM and keep small system disks (excepts for db/web usage, which are not huge in general).
That's all.
edit: is it more clear now?
Let's see if I reword it correctly....
If your VM needs a lot of file storage.... then it is better to mount that from a file server rather than keeping it in the original VM?
Yup, that's it. Because a lot of file storage will mean a large VDI, which is "dangerous".
-
@olivier said in Someone doesn't like local storage for large amounts of data:
@scottalanmiller said in Someone doesn't like local storage for large amounts of data:
@olivier said in Someone doesn't like local storage for large amounts of data:
The thing, initially, was about having VMs with large VDIs. Which is for me not a good practice.
But if you need to store a large amount of data, it's better to connect to a remote file share in the VM and keep small system disks (excepts for db/web usage, which are not huge in general).
That's all.
edit: is it more clear now?
Let's see if I reword it correctly....
If your VM needs a lot of file storage.... then it is better to mount that from a file server rather than keeping it in the original VM?
Yup, that's it. Because a lot of file storage will mean a large VDI, which is "dangerous".
Okay, that part makes sense
-
\o/
-
I'm blaming @Dashrender for that one.
-
I'm blaming myself for doing multiple things at once. Got a trip early tomorrow, so I'm going to bed
See ya! -
@olivier said in Someone doesn't like local storage for large amounts of data:
I'm blaming myself for doing multiple things at once. Got a trip early tomorrow, so I'm going to bed
See ya!Good night.
-
@scottalanmiller said in Someone doesn't like local storage for large amounts of data:
@olivier said in Someone doesn't like local storage for large amounts of data:
The thing, initially, was about having VMs with large VDIs. Which is for me not a good practice.
But if you need to store a large amount of data, it's better to connect to a remote file share in the VM and keep small system disks (excepts for db/web usage, which are not huge in general).
That's all.
edit: is it more clear now?
Let's see if I reword it correctly....
If your VM needs a lot of file storage.... then it is better to mount that from a file server rather than keeping it in the original VM?
Ok, I get that, but this goes against the "new fangled" HCI (call it what you want) use of local "attached" storage?
-
@FATeknollogee said in Someone doesn't like local storage for large amounts of data:
@scottalanmiller said in Someone doesn't like local storage for large amounts of data:
@olivier said in Someone doesn't like local storage for large amounts of data:
The thing, initially, was about having VMs with large VDIs. Which is for me not a good practice.
But if you need to store a large amount of data, it's better to connect to a remote file share in the VM and keep small system disks (excepts for db/web usage, which are not huge in general).
That's all.
edit: is it more clear now?
Let's see if I reword it correctly....
If your VM needs a lot of file storage.... then it is better to mount that from a file server rather than keeping it in the original VM?
Ok, I get that, but this goes against the "new fangled" HCI (call it what you want) use of local "attached" storage?
It doesn't not really. That's what caught @Dashrender it's more two things...
- Split up workloads to keep size down of individual loads
- Resort to raw storage when containerized storage gets too large and the above cannot be actioned
-
@scottalanmiller said in Someone doesn't like local storage for large amounts of data:
I'm blaming @Dashrender for that one.
Whatever
-
@scottalanmiller said in Someone doesn't like local storage for large amounts of data:
@FATeknollogee said in Someone doesn't like local storage for large amounts of data:
@scottalanmiller said in Someone doesn't like local storage for large amounts of data:
@olivier said in Someone doesn't like local storage for large amounts of data:
The thing, initially, was about having VMs with large VDIs. Which is for me not a good practice.
But if you need to store a large amount of data, it's better to connect to a remote file share in the VM and keep small system disks (excepts for db/web usage, which are not huge in general).
That's all.
edit: is it more clear now?
Let's see if I reword it correctly....
If your VM needs a lot of file storage.... then it is better to mount that from a file server rather than keeping it in the original VM?
Ok, I get that, but this goes against the "new fangled" HCI (call it what you want) use of local "attached" storage?
It doesn't not really. That's what caught @Dashrender it's more two things...
- Split up workloads to keep size down of individual loads
- Resort to raw storage when containerized storage gets too large and the above cannot be actioned
what does resort to raw storage mean?
-
@Dashrender said in Someone doesn't like local storage for large amounts of data:
@scottalanmiller said in Someone doesn't like local storage for large amounts of data:
@FATeknollogee said in Someone doesn't like local storage for large amounts of data:
@scottalanmiller said in Someone doesn't like local storage for large amounts of data:
@olivier said in Someone doesn't like local storage for large amounts of data:
The thing, initially, was about having VMs with large VDIs. Which is for me not a good practice.
But if you need to store a large amount of data, it's better to connect to a remote file share in the VM and keep small system disks (excepts for db/web usage, which are not huge in general).
That's all.
edit: is it more clear now?
Let's see if I reword it correctly....
If your VM needs a lot of file storage.... then it is better to mount that from a file server rather than keeping it in the original VM?
Ok, I get that, but this goes against the "new fangled" HCI (call it what you want) use of local "attached" storage?
It doesn't not really. That's what caught @Dashrender it's more two things...
- Split up workloads to keep size down of individual loads
- Resort to raw storage when containerized storage gets too large and the above cannot be actioned
what does resort to raw storage mean?
Use direct access to the storage rather than a VDI. The size of the VDI is the concern.
So Xen as an example, you can use a raw LVM partition for a VM rather than VDI file. This fixes the large VDI problem.
A typical setup would be to have one smaller VDI, say 20GB, for the OS and then a raw partition, say 30TB, for the files.
-
@scottalanmiller said in Someone doesn't like local storage for large amounts of data:
@Dashrender said in Someone doesn't like local storage for large amounts of data:
@scottalanmiller said in Someone doesn't like local storage for large amounts of data:
@FATeknollogee said in Someone doesn't like local storage for large amounts of data:
@scottalanmiller said in Someone doesn't like local storage for large amounts of data:
@olivier said in Someone doesn't like local storage for large amounts of data:
The thing, initially, was about having VMs with large VDIs. Which is for me not a good practice.
But if you need to store a large amount of data, it's better to connect to a remote file share in the VM and keep small system disks (excepts for db/web usage, which are not huge in general).
That's all.
edit: is it more clear now?
Let's see if I reword it correctly....
If your VM needs a lot of file storage.... then it is better to mount that from a file server rather than keeping it in the original VM?
Ok, I get that, but this goes against the "new fangled" HCI (call it what you want) use of local "attached" storage?
It doesn't not really. That's what caught @Dashrender it's more two things...
- Split up workloads to keep size down of individual loads
- Resort to raw storage when containerized storage gets too large and the above cannot be actioned
what does resort to raw storage mean?
Use direct access to the storage rather than a VDI. The size of the VDI is the concern.
So Xen as an example, you can use a raw LVM partition for a VM rather than VDI file. This fixes the large VDI problem.
A typical setup would be to have one smaller VDI, say 20GB, for the OS and then a raw partition, say 30TB, for the files.
How does this make the situation any better? It still takes hours to migrate that data from one host to another. Does being raw somehow enable faster access to that data?
