Solved Get Large Disk Images on XenServer 6.5 on Local Filesystem?
-
@scottalanmiller But why not make a file server with several vDisk rather than a massive disk?
What are the benefits?
-
@DustinB3403 said in Get Large Disk Images on XenServer 6.5 on Local Filesystem?:
@ntoxicator The issue here (does present a single point of failure) but only for the storage device.
Sure if that device dies that storage is offline, but the VM is still usable. So it's a Storage SPOF, rather than a System SPOF.
I'd rather have a Storage SPOF (in this case) than a System SPOF if I had the choice. Which to alleviate this SPOF you'd get a good NAS/SAN and use that.
Not a unreliable piece of garbage.
What's the difference in SPOF, though, either one fails your services are down. Total outage either way.
-
I disagree, you have an outage of a single storage platform. If everything is hosted from this external storage, sure then it could be a complete outage.
But you'd still have access to the VM to manage / repair whatever is broken at the VM level (if anything).
-
@DustinB3403 said in Get Large Disk Images on XenServer 6.5 on Local Filesystem?:
I disagree, you have an outage of a single storage platform. If everything is hosted from this external storage, sure then it could be a complete outage.
But you'd still have access to the VM to manage / repair whatever is broken at the VM level (if anything).
So specifically in a case where you have storage that isn't important? Sure, like if you are mounting a backup share and are happy losing it. Certainly doesn't apply to any normal use case. If your database vanishes from your database server, "fixing" the VM is pointless. If the storage is gone from your file server, keeping it up and running is pointless.
What scenario did you have in mind?
-
@scottalanmiller I'm just trying to think of a scenario where you need to have a larger than 2TB partition for file services. (I haven't seen any in my experience - don't take that as me saying they don't exist).
Which if you need a larger partition, the current solution is external storage to the Hypervisor (yes it sucks for all of the reasons mentioned and going through your head).
The only cases that I could imagine this as being used would be if you wanted to attach an device to a VM for backup purposes, which then get pushed off. Massive "localish" storage for the VM to quickly replicate from the proper shares to the iSCSI device.
-
@DustinB3403 said in Get Large Disk Images on XenServer 6.5 on Local Filesystem?:
@scottalanmiller I'm just trying to think of a scenario where you need to have a larger than 2TB partition for file services.
This is super common for a business of any size. Pretty much, if you have 2TB of files, you have run into this scenario.
-
Other than tiny file servers, what situations can you imagine where a single large partition is not desired? Try it in reverse. You seem to think this is uncommon, but in reality, I think it's the standard case.
-
I've seen far more smaller partitions (2TB and under) shared out than I've seen massive (+2TB) shares configured and setup. .
The burden to provide examples isn't on me, but on you @scottalanmiller.
-
@scottalanmiller I work in about the smallest environment it's possible to still have IT, and only because 5 different companies are actually under the same ownership. Even I'm looking at that 2TB cap saying, that's just not enough!
-
@DustinB3403 said in Get Large Disk Images on XenServer 6.5 on Local Filesystem?:
I've seen far more smaller partitions (2TB and under) shared out than I've seen massive (+2TB) shares configured and setup. .
The burden to provide examples isn't on me, but on you @scottalanmiller.
We have home directories that mount from multiple servers that have 52 TB a piece.
-
@DustinB3403 said in Get Large Disk Images on XenServer 6.5 on Local Filesystem?:
I've seen far more smaller partitions (2TB and under) shared out than I've seen massive (+2TB) shares configured and setup. .
The burden to provide examples isn't on me, but on you @scottalanmiller.
Not really, I asked for an industry standard, you are saying that it's not a real use case. Since just about every company of any size is looking at scores or hundreds of TB of storage (or PB even), saying that 2TB is a reasonable upper bounds needs some explaining. Why would this be? Other than very tiny companies or those with very tiny storage needs, when would so little storage capacity make sense?
Considering none of my storage units at home are this small, not even from six years ago or more, this makes no sense to me. From a size perspective, this is below the home line in many cases... just storing music, home movies and such often requires far more than this (I have about 12TB for home.)
-
The fact that Gluster, CEPH, Exablox, Scale and others are vendors dealing specifically with these kinds of limits, but at more like the 2PB, not 2TB, scale, I think we are way past needing to show why 2TB is a small limit.
-
@DustinB3403 said in Get Large Disk Images on XenServer 6.5 on Local Filesystem?:
I've seen far more smaller partitions (2TB and under) shared out than I've seen massive (+2TB) shares configured and setup. .
The burden to provide examples isn't on me, but on you @scottalanmiller.
Also, 2TB is not massive by any means. That's a ~$50 consumer drive. I'm thinking of how you argued with robinhood on SpiceWorks how multiple PB of data should be virtualized. How do you plan on doing that here?
-
So back full circle.
Any idea's how to circumvent this?
As we can have HUGE locaclized storage repository for XenServer, but the Guest (Windows server VM) will only allow 2TB due to Windows VHD limitations...
so for those of us running large file servers (myself). this 2TB is an issue; its too small!
So would have to pool the disks together?
or rely on some large external storage device with NFS/SMB shares? This just adds to the infrastructure costs and possible failure points.
@Scale computing nodes...... same limitations I presume?
-
@scottalanmiller said in Get Large Disk Images on XenServer 6.5 on Local Filesystem?:
just storing music, home movies and such often requires far more than this (I have about 12TB for home.)
Ya, I have almost 2TB just in music.
-
I'm not saying that 2TB or 2PB shouldn't be virtual, I'm saying that the amount presented out shouldn't need to be larger than 2TB (per share) for a virtual machine for the very reasons of usability and restore options.
This is data administration, not system design. You have 2TB shares, if you need larger use another method. RobinHood specifically believes that you should never virtualize your file systems or work-loads.
I'm specifically saying present multiple 2TB shares out, unless you need more, in which case use an iSCSI target.
-
@ntoxicator said in Get Large Disk Images on XenServer 6.5 on Local Filesystem?:
So back full circle.
Any idea's how to circumvent this?
As we can have HUGE locaclized storage repository for XenServer, but the Guest (Windows server VM) will only allow 2TB due to Windows VHD limitations...
so for those of us running large file servers (myself). this 2TB is an issue; its too small!
So would have to pool the disks together?
or rely on some large external storage device with NFS/SMB shares? This just adds to the infrastructure costs and possible failure points.
@Scale computing nodes...... same limitations I presume?
You could switch to Xen and use a real image file. Scale won't have this limitation. It's KVM, which I'm also running, and with qcow2 the limit is something like 7 exabytes.
-
@DustinB3403 said in Get Large Disk Images on XenServer 6.5 on Local Filesystem?:
I'm not saying that 2TB or 2PB shouldn't be virtual, I'm saying that the amount presented out shouldn't need to be larger than 2TB (per share) for a virtual machine for the very reasons of usability and restore options.
I don't understand this at all. Why would you have benefits to small chunks of a single filesystem? I've heard this before but never heard of a reason for it.
Presented out obviously has to be larger than 2TB, that's not an option. That the parts that make up the large share should be made up of tiny pieces is standardly considered a mistake of 2005 era SAN design. Why do you feel that this industry accepted mistake of a decade ago should be made standard again today?
-
in which case use an iSCSI target.
So you're saying to use Microsofts iSCSI initiator to connect a disk? I've been hit over the head before to suggesting that.
-
@johnhooks said in Get Large Disk Images on XenServer 6.5 on Local Filesystem?:
You could switch to Xen and use a real image file. Scale won't have this limitation. It's KVM, which I'm also running, and with qcow2 the limit is something like 7 exabytes.
Oh I know, I can make a pretty epic file server on the Scale HC3
It's way ahead for that. This is not for a system that I manage that I was looking for the answer. Yes, "real" Xen without XS limitations does this without a problem. And Scale HC3 does it without thinking. Why XS is introducing this problem is beyond me.
