Connecting a NAS or SAN to a VMWare host
-
OK I'm having a discussion with a colleague and I guess I'm missing something.
How much bandwidth is needed between a NAS or SAN to run say an SQL server on a VMWare host over ISCSI?
I realize you need a whole host of information, like how many IOPs are needed, and much data is moving - I can't give that to you.. so let's just start with a starting point.. you would never do less than what?
Now let's ramp that up. 3 blades (in an IBM Blade Center) connected to that same NAS - running 26 windows VMs - now how much bandwidth should you have? (i.e. how many 1 gb connections?
-
Generally... relatively little. What you care most about is having a low latency. The bandwidth is rarely an issue. You can never know for sure without measuring (DPACK or whatever) but you can run a lot of IOPS over a single GigE iSCSI channel. Throughput is less of a concern, especially for databases. IOPS are the bigger concern.
-
You would never do less than: GigE
GigE is definitely the lowest speed channel that I would give for a database's storage backbone. That could be iSCSI, NFS, FC, whatever.
-
@Dashrender said:
Now let's ramp that up. 3 blades (in an IBM Blade Center) connected to that same NAS - running 26 windows VMs - now how much bandwidth should you have? (i.e. how many 1 gb connections?
Remember that this is purely hypothetical what you are asking. Never, ever would I have a NAS or a SAN backing VMs in anything less than four nodes, ever (in production.) Slow and fragile while being expensive. That's a horrible combination.
Really the slowest you should ever have is 6Gb/s because anything else, even things that are slower like GigE, is more expensive without benefit.
-
Number of VMs and number of nodes and all of that might give us a tiny insight into IOPS, but only the tiniest. But bandwidth, that's really, really hard. IOPS tend to stick within a 100x variation or so (100 IOPS on the low end, 10,000 IOPS on the high end) but bandwidth would be much bigger with less than 100Mb/s on the low end and 12Gb/s on the high end. The fluctuations are bigger and the predictability is worse.
-
My friend's current setup in a 'hosted' solution is currently 26 physical servers.
I say 'hosted' because it sounds like the hosting company is a home grown extension of what used to the company my friend works for.
Anyhow - He's looking at another real hosted solution where they are providing 3 IBM blades in a bladecenter. Each blade has 2 attached disks (for the OS I'm guessing) but not for the VMs. His proposal is to install a NAS (IBM V7000 - I think) and connect those to the blades.
Should he just walk away from that solution?
-
@scottalanmiller said:
Number of VMs and number of nodes and all of that might give us a tiny insight into IOPS, but only the tiniest. But bandwidth, that's really, really hard. IOPS tend to stick within a 100x variation or so (100 IOPS on the low end, 10,000 IOPS on the high end) but bandwidth would be much bigger with less than 100Mb/s on the low end and 12Gb/s on the high end. The fluctuations are bigger and the predictability is worse.
What caps IOPs? Is there a max you can pipe through a 1 GbE connection?
-
@Dashrender said:
Anyhow - He's looking at another real hosted solution where they are providing 3 IBM blades in a bladecenter. Each blade has 2 attached disks (for the OS I'm guessing) but not for the VMs.
This is partially why I hate blades. They not only have problems on their own (more fragile, too expensive) but they then lead to second and third order bad decision making. The blades on their own might not be a horrible decision (but they probably are) but because of that one bad decision, then a second bad decision of needing high cost, fragile, complex shared storage is suddenly needed.
-
@Dashrender said:
Should he just walk away from that solution?
Not, not out of hand. But he needs to be wary. It's a silly setup top to bottom and probably way too costly for what he is getting.
-
@Dashrender said:
Each blade has 2 attached disks (for the OS I'm guessing) but not for the VMs.
That is the intention of the design, correct.
-
@Dashrender said:
What caps IOPs? Is there a max you can pipe through a 1 GbE connection?
There is probably a theoretical maximum but it would be really, really high. Your needs need to be determined by a combination of your IOPS and your throughput needs. Your IOPS are "almost" purely determined by your disk array. Your throughput is primarily determined by your connection medium. So to make any kind of determination you really need a complete picture, which is not easy to get. That's why a lot of people just go with 8Gb/s fiber channel, 10GigE NFS or iSCSI or 6-12Gb/s SAS. Just overbuy and not worry about it.
-
IOPS are throughput are only kind of tied together. But in practice, the higher the IOPS the higher the throughput. Each IOP, by definition, requires some amount of bandwidth, but it might be a pretty small amount.
Some workloads, like databases, typically are huge IOPS with very small throughput.
On the opposite side are fileservers which are typically tiny IOPS with huge throughput.
-
Sorry to sidetrack for a second, but I may be misunderstanding IOPS a little bit. Does the bit-size of an IO function vary, or is it 1-bit-per-I/O operation?
-
@art_of_shred said:
Sorry to sidetrack for a second, but I may be misunderstanding IOPS a little bit. Does the bit-size of an IO function vary, or is it 1-bit-per-I/O operation?
Correct, the "payload" of an IOP can be all over the place. An IOP would be a single SCSI or ATA command (operation) which might be really simple and tiny or it might be rather sizable. So while throughput could be roughly calculated as IOP x IOP Size, the IOP Size will vary per IOP so you would need to know the average size for a specific workload to have a good idea of the throughput.
-
OK, got it... but stop calling it an IOP. It's an Input/Output, not an Input/output Per. (sorry :P)
-
@art_of_shred said:
OK, got it... but stop calling it an IOP. It's an Input/Output, not an Input/output Per. (sorry :P)
LOL, that's correct obviously. It's pronounced that way, though, because people are thinking of "Input / Output Operation", the "Op" of operation makes the letters feel that way in your head. So people say IOP all of the time.
Here is the calculation as Wikipedia writes it...
IOPS * TransferSizeInBytes = BytesPerSec
-
@art_of_shred said:
OK, got it... but stop calling it an IOP. It's an Input/Output, not an Input/output Per. (sorry :P)
Lol Yup, it's either IO or IOPS. One is the actual thing, the other is a measurement.
-
@ajstringham said:
@art_of_shred said:
OK, got it... but stop calling it an IOP. It's an Input/Output, not an Input/output Per. (sorry :P)
Lol Yup, it's either IO or IOPS. One is the actual thing, the other is a measurement.
Sort of. IOPS is actually sort for "Input / Output Operations Per Second." So which letters stand for which words?
-
IOOP... Aye-Oop!
-
@scottalanmiller said:
@ajstringham said:
@art_of_shred said:
OK, got it... but stop calling it an IOP. It's an Input/Output, not an Input/output Per. (sorry :P)
Lol Yup, it's either IO or IOPS. One is the actual thing, the other is a measurement.
Sort of. IOPS is actually sort for "Input / Output Operations Per Second." So which letters stand for which words?
Never saw it with "operations" in there. And if so, where is the other "O"?
