Vendor Mistake - VMware Infrastructure Decisions



  • Before I started here a couple of months ago, my boss purchased a couple of Dell R630s and a PowerVault MD3820i (20 drive bays) to be our new infrastructure at HQ. We have dual 10Gb PowerConnect switches and two UPS devices, each connected to a different circuit. The plan is to rebuild the infrastructure on vSphere Standard (licenses already purchased) and have a similar setup in a datacenter somewhere (replicate the SANs, etc.). We're using AppAssure for backups (again, already purchased).

    The PowerVault has 16 SAS drives that are 1.8 TB 7200 RPM SED drives and 4 SAS drives that are 400 GB SSD for caching. Well, we made disk groups and virtual disks using the SEDs (letting the SAN manage the keys), but it turns out we cannot use the SSDs they sent us for caching. In fact, they don't have SED SSDs for this model SAN.

    At the time the sale was made, Dell ensured my boss everything would work as he requested (being able to use the SSDs for caching with the 7200 RPM SED drives). Now that we know this isn't going to be the case, we have some options.

    First, they recommended we trade in the PowerVault for a Compellent and Equalogic. The boss did not want that because he was saying you are forced to do RAID 6 on those devices and cannot go with RAID 10 in your disk groups. As another option, Dell recommended we put the SSDs in our two hosts and use Infinio so we can do caching with the drives we have. In this case we would make Dell pay for the Infinio licenses and possibly more RAM since they made the mistake.

    But I'm wondering if perhaps there is another option. Each server has 6 drive bays. So we have 20 drives total. Couldn't we have Dell take the SAN back, give us another R630, and pay for licenses of VMware vSAN for all 3 hosts? Each server has four 10 Gb NICs and two 1 Gb NICs. That might require we get additional NICs. But in this case, I'm not sure drive encryption is an option or if we can utilize the SEDs at all.

    I've not double-checked the vSAN HCL or anything for the gear in our servers as this is just me spit balling. Is there some other option we have not considered? We're looking to get the 14 TB or so of usable space that RAID 10 will provide, but the self-encrypting drives were deemed a necessity by the boss. And without some type of caching, we will not hit our IOPs requirements.

    Any advice is much appreciated.


  • Service Provider

    Returning the SAN and picking up a third host seems to be a step in the right direction. Your storage space will be close to what you would have had with the SAN.

    The questions I have is what is your backup target and will AppAssure work with 3 local hosts and your target? You mentioned needing cache to hit your IOP target. Will the 3 hosts and direct attached make it? Years ago at the last VMUG conference I went to there were a ton of vendors that had caching cards for servers to help out with direct attached loads. With the money you get back from the SAN, that could be a wash.



  • I'd recommend watching @scottalanmiller's MangoCon Talk on architectures.

    I never would have considered a SAN in the first place at two hosts.. so yeah. 3 hosts and VSAN.


  • Service Provider

    I haven't found a case where the cost of vSAN made sense with so many other options out there. I think with 3 hosts and 2 CPUs each he would be in the neighborhood of $14,000 for the vSAN licenses. Has anyone found a good use case for vSAN?



  • @Mike-Davis said in Vendor Mistake - VMware Infrastructure Decisions:

    I haven't found a case where the cost of vSAN made sense with so many other options out there. I think with 3 hosts and 2 CPUs each he would be in the neighborhood of $14,000 for the vSAN licenses. Has anyone found a good use case for vSAN?

    Really? Geesh, nevermind, XS7 + gluster.


  • Service Provider

    @Mike-Davis said in Vendor Mistake - VMware Infrastructure Decisions:

    I haven't found a case where the cost of vSAN made sense with so many other options out there. I think with 3 hosts and 2 CPUs each he would be in the neighborhood of $14,000 for the vSAN licenses. Has anyone found a good use case for vSAN?

    Well, since you get get HA from a SAN at that price, if you want a VMware product that price is great. I'm not sure what you are saying. Have you tried pricing the alternative? I've not found a case where VSAN hasn't been the clear winner.


  • Service Provider

    @travisdh1 said in Vendor Mistake - VMware Infrastructure Decisions:

    @Mike-Davis said in Vendor Mistake - VMware Infrastructure Decisions:

    I haven't found a case where the cost of vSAN made sense with so many other options out there. I think with 3 hosts and 2 CPUs each he would be in the neighborhood of $14,000 for the vSAN licenses. Has anyone found a good use case for vSAN?

    Really? Geesh, nevermind, XS7 + gluster.

    If you are a shop choosing the VMware support cost already, that's nothing. You have to keep it in perspective. If you are running VMware here it means that at a minimum you have already committed to Essentials Plus which really gives you nothing (other than support, that's what you are paying for) that XS or Hyper-V don't do for free. If you blink at $14K for VSAN, something is very wrong elsewhere. And compared to a SAN equivalent, it is about half the price.


  • Service Provider

    @Mike-Davis said in Vendor Mistake - VMware Infrastructure Decisions:

    I haven't found a case where the cost of vSAN made sense with so many other options out there. I think with 3 hosts and 2 CPUs each he would be in the neighborhood of $14,000 for the vSAN licenses. Has anyone found a good use case for vSAN?

    Compare to the SAN alternative:

    https://mangolassi.it/topic/11937/cost-study-3-node-scale-vs-3-node-vmware-ipod/
    https://mangolassi.it/topic/11936/cost-study-3-node-scale-vs-3-node-vmware-vsan/

    VSAN beats the IPOD SAN by $30,000 while providing a better solution.


  • Service Provider

    @NetworkNerd said in Vendor Mistake - VMware Infrastructure Decisions:

    Any advice is much appreciated.

    You only need two nodes, no need to change that. Tell Dell to take back the gear that they wrongfully sold to you and to provide you with the disks that you need for local storage and give back the difference in a check . Then get Starwind VSAN which only requires two nodes. You have no need for the third node, nor the SAN, nor the VMware VSAN cost. All of that is just "extra."



  • @Mike-Davis said in Vendor Mistake - VMware Infrastructure Decisions:

    The questions I have is what is your backup target and will AppAssure work with 3 local hosts and your target? You mentioned needing cache to hit your IOP target. Will the 3 hosts and direct attached make it? Years ago at the last VMUG conference I went to there were a ton of vendors that had caching cards for servers to help out with direct attached loads. With the money you get back from the SAN, that could be a wash.

    The backup target will be the AppAssure appliance. It has internal storage of somewhere around 18 TB I believe. This will be replicated to another appliance in a datacenter offsite.



  • @scottalanmiller said in Vendor Mistake - VMware Infrastructure Decisions:

    @NetworkNerd said in Vendor Mistake - VMware Infrastructure Decisions:

    Any advice is much appreciated.

    You only need two nodes, no need to change that. Tell Dell to take back the gear that they wrongfully sold to you and to provide you with the disks that you need for local storage and give back the difference in a check . Then get Starwind VSAN which only requires two nodes. You have no need for the third node, nor the SAN, nor the VMware VSAN cost. All of that is just "extra."

    That's certainly an option.  If I'm reading this correctly, Starwind has an option to run in-kernel on VMware - https://www.starwindsoftware.com/whitepapers/free-vs-paid.pdf.  It looks like this would also allow caching to SSDs or to RAM within the host.  So I guess you just choose the drives you want to be part of your storage and those you want to be used for caching and go from there?  And since we'd be looking at the iSCSI HA, is your total usable storage only the storage that is in one of the servers?


  • Service Provider

    @NetworkNerd said in Vendor Mistake - VMware Infrastructure Decisions:

    @scottalanmiller said in Vendor Mistake - VMware Infrastructure Decisions:

    @NetworkNerd said in Vendor Mistake - VMware Infrastructure Decisions:

    Any advice is much appreciated.

    You only need two nodes, no need to change that. Tell Dell to take back the gear that they wrongfully sold to you and to provide you with the disks that you need for local storage and give back the difference in a check . Then get Starwind VSAN which only requires two nodes. You have no need for the third node, nor the SAN, nor the VMware VSAN cost. All of that is just "extra."

    That's certainly an option.  If I'm reading this correctly, Starwind has an option to run in-kernel on VMware - https://www.starwindsoftware.com/whitepapers/free-vs-paid.pdf.  It looks like this would also allow caching to SSDs or to RAM within the host.  So I guess you just choose the drives you want to be part of your storage and those you want to be used for caching and go from there?  And since we'd be looking at the iSCSI HA, is your total usable storage only the storage that is in one of the servers?

    Paging @kooler on that one.


  • Service Provider

    @NetworkNerd said in Vendor Mistake - VMware Infrastructure Decisions:

    And since we'd be looking at the iSCSI HA, is your total usable storage only the storage that is in one of the servers?

    That is necessary for HA to be possible. Whether this is a SAN, VMware VSAN, Starwind VSAN, NAS.... if you don't replicate it, it's not really HA. So yes, the data has to be on both hosts as there are only two, they have to be mirrored.


  • Service Provider

    With only 2 hosts that have 6 bays for drives, he's going to need direct attached storage. What does Dell offer for this?


  • Service Provider

    What storage is needed? This is the problem with buying 1U servers, the decision to be dependent on external storage was used to make sure that these don't meet the need either. Mike is right, tell Dell to take it ALL back and replace them with R730 instead. That will solve the issue.



  • @Mike-Davis said in Vendor Mistake - VMware Infrastructure Decisions:

    With only 2 hosts that have 6 bays for drives, he's going to need direct attached storage. What does Dell offer for this?

    Sorry - the hosts have 8 bays for drives, but we would fill 1-2 for each host with SSDs for caching (ideally).


  • Service Provider

    @NetworkNerd said in Vendor Mistake - VMware Infrastructure Decisions:

    @Mike-Davis said in Vendor Mistake - VMware Infrastructure Decisions:

    With only 2 hosts that have 6 bays for drives, he's going to need direct attached storage. What does Dell offer for this?

    Sorry - the hosts have 8 bays for drives, but we would fill 1-2 for each host with SSDs for caching (ideally).

    Would 2 + 6 be enough for your workload? Doesn't seem likely.



  • I know there's pressure from above to get something in place quickly because the gear has been sitting for a bit since it was ordered (short on man power before I came here in December). I believe Dell wants to replace hardware with hardware (EMC Unity, Compellent, etc.) so they can stick it to us on maintenance costs of the SAN over time and then try to sell us another one someday. With them taking the PowerVault back and us going with some some type of VSAN solution (Starwind or other), I don't think their margins are as high over time.


  • Service Provider

    @NetworkNerd said in Vendor Mistake - VMware Infrastructure Decisions:

    I know there's pressure from above to get something in place quickly because the gear has been sitting for a bit since it was ordered (short on man power before I came here in December). I believe Dell wants to replace hardware with hardware (EMC Unity, Compellent, etc.) so they can stick it to us on maintenance costs of the SAN over time and then try to sell us another one someday. With them taking the PowerVault back and us going with some some type of VSAN solution (Starwind or other), I don't think their margins are as high over time.

    Yeah, not even close. You could always just cancel the whole deal and move to another vendor, too. If Dell gives you any pressure, send it all back. The sale was in bad faith, you shouldn't be responsible for "fixing it."

    They should be happy that you are willing to work with them at all at this point. They should be bending over backwards to not lose a customer.



  • @scottalanmiller said in Vendor Mistake - VMware Infrastructure Decisions:

    @NetworkNerd said in Vendor Mistake - VMware Infrastructure Decisions:

    I know there's pressure from above to get something in place quickly because the gear has been sitting for a bit since it was ordered (short on man power before I came here in December). I believe Dell wants to replace hardware with hardware (EMC Unity, Compellent, etc.) so they can stick it to us on maintenance costs of the SAN over time and then try to sell us another one someday. With them taking the PowerVault back and us going with some some type of VSAN solution (Starwind or other), I don't think their margins are as high over time.

    Yeah, not even close. You could always just cancel the whole deal and move to another vendor, too. If Dell gives you any pressure, send it all back. The sale was in bad faith, you shouldn't be responsible for "fixing it."

    They should be happy that you are willing to work with them at all at this point. They should be bending over backwards to not lose a customer.

    They have not pressured us at all (quite the other way around, actually). From what I have heard the account team say (even after not having been on all the calls), they seem to truly want to do what it takes to make things right and retain a happy customer at the end of the day. But I did mention to the boss that I found it extremely interesting they never offered a VSAN type solution even as an option when looking at "alternatives."


  • Service Provider

    @NetworkNerd said in Vendor Mistake - VMware Infrastructure Decisions:

    @scottalanmiller said in Vendor Mistake - VMware Infrastructure Decisions:

    @NetworkNerd said in Vendor Mistake - VMware Infrastructure Decisions:

    I know there's pressure from above to get something in place quickly because the gear has been sitting for a bit since it was ordered (short on man power before I came here in December). I believe Dell wants to replace hardware with hardware (EMC Unity, Compellent, etc.) so they can stick it to us on maintenance costs of the SAN over time and then try to sell us another one someday. With them taking the PowerVault back and us going with some some type of VSAN solution (Starwind or other), I don't think their margins are as high over time.

    Yeah, not even close. You could always just cancel the whole deal and move to another vendor, too. If Dell gives you any pressure, send it all back. The sale was in bad faith, you shouldn't be responsible for "fixing it."

    They should be happy that you are willing to work with them at all at this point. They should be bending over backwards to not lose a customer.

    They have not pressured us at all (quite the other way around, actually). From what I have heard the account team say (even after not having been on all the calls), they seem to truly want to do what it takes to make things right and retain a happy customer at the end of the day. But I did mention to the boss that I found it extremely interesting they never offered a VSAN type solution even as an option when looking at "alternatives."

    Well, likely they don't offer it because they don't make it. Even fixing a sale you don't expect someone to switch to a competitors product. Now, technically, VMware VSAN is part of Dell so it is their solution to sell. But they are a separate operating company and the Dell team probably doesn't see it as a single entity.



  • @scottalanmiller said in Vendor Mistake - VMware Infrastructure Decisions:

    Well, likely they don't offer it because they don't make it. Even fixing a sale you don't expect someone to switch to a competitors product. Now, technically, VMware VSAN is part of Dell so it is their solution to sell. But they are a separate operating company and the Dell team probably doesn't see it as a single entity.

    They were the ones who suggested Infinio, and I don't even think they sell it. But I see what you mean.


  • Vendor

    @scottalanmiller If your doing a 3 node vSAN for a low cost deployment you should go single socket and get more core's per proc. Leaves you room to scale later and costs the vSAN cost in half.

    Also that cost study on vSAN is funky. The costs don't make sense to me based on quotes I've seen (I suspect no one actually was trying to get a discounted quote, and put 5 years or support or something on it). It also uses SATA drives (not certified for vSAN) for capacity instead of NL-SAS drives, and looks to be using a non-certified cache tier drive.


  • Vendor

    @scottalanmiller They have a different compensation plan (although it changed as we are now in FY18 for Dell's calendar year) so you'll see different behaviors in their sales force from different people.


  • Service Provider

    @John-Nicholson said in Vendor Mistake - VMware Infrastructure Decisions:

    Also that cost study on vSAN is funky. The costs don't make sense to me based on quotes I've seen (I suspect no one actually was trying to get a discounted quote, and put 5 years or support or something on it). It also uses SATA drives (not certified for vSAN) for capacity instead of NL-SAS drives, and looks to be using a non-certified cache tier drive.

    Listed MSRP in both cases. So neither side uses a discount.


  • Service Provider

    @John-Nicholson said in Vendor Mistake - VMware Infrastructure Decisions:

    @scottalanmiller If your doing a 3 node vSAN for a low cost deployment you should go single socket and get more core's per proc. Leaves you room to scale later and costs the vSAN cost in half.

    They are likely stuck here with whatever was already bought. But good info for a greenfield deployment. Or if they manage to return these for three R730 for example.



  • Honestly, I would return EVERYTHING.

    Then I would sit down and design it the right way, using a few R730xd servers, with appropriate specs to accommodate your needs. With that and Starwind vSAN, you can get your HA.

    Do you actually need HA? Does the company feel spending the money for real HA is a business requirement and makes financial sense?


  • Service Provider

    Tim has a point. At some point you need to start over. This is the core of the business, right? Does anyone really want the core of the business to be a huge compromise based on "fixing" layer after layer of bad decisions before? The design wasn't the best, the products weren't right, the products that are okay were based on products that were not, etc. Start over, do it right the whole way. Go to management, explain that this isn't a place where you rush or compromise - this has to work.



  • @scottalanmiller said in Vendor Mistake - VMware Infrastructure Decisions:

    @John-Nicholson said in Vendor Mistake - VMware Infrastructure Decisions:

    @scottalanmiller If your doing a 3 node vSAN for a low cost deployment you should go single socket and get more core's per proc. Leaves you room to scale later and costs the vSAN cost in half.

    They are likely stuck here with whatever was already bought. But good info for a greenfield deployment. Or if they manage to return these for three R730 for example.

    I'm not entirely certain we'll be stuck with what we bought. My boss and I were on a conference call with folks from Dell yesterday afternoon. They were talking about different options in SAN devices that would meet our requirements (whether it was Compellent, EMC, etc.), but the biggest issue was that these options were so expensive. Again, not one of them mentioned the potential for a VSAN deployment, so we brought it up (using either VMware VSAN or Starwind). The Dell team has to go back and redesign a quote for gear that would better support a VSAN deployment. In their words, they would likely have to return the servers and the PowerVault we have right now (not sure about the other gear - PowerConnect switches, TrippLite devices, APC PDUs, AppAssure appliance, and ip KVM switch).

    I'll be curious to see what comes back when they re-quote.


  • Service Provider

    @NetworkNerd said in Vendor Mistake - VMware Infrastructure Decisions:

    @scottalanmiller said in Vendor Mistake - VMware Infrastructure Decisions:

    @John-Nicholson said in Vendor Mistake - VMware Infrastructure Decisions:

    @scottalanmiller If your doing a 3 node vSAN for a low cost deployment you should go single socket and get more core's per proc. Leaves you room to scale later and costs the vSAN cost in half.

    They are likely stuck here with whatever was already bought. But good info for a greenfield deployment. Or if they manage to return these for three R730 for example.

    I'm not entirely certain we'll be stuck with what we bought. My boss and I were on a conference call with folks from Dell yesterday afternoon. They were talking about different options in SAN devices that would meet our requirements (whether it was Compellent, EMC, etc.), but the biggest issue was that these options were so expensive. Again, not one of them mentioned the potential for a VSAN deployment, so we brought it up (using either VMware VSAN or Starwind). The Dell team has to go back and redesign a quote for gear that would better support a VSAN deployment. In their words, they would likely have to return the servers and the PowerVault we have right now (not sure about the other gear - PowerConnect switches, TrippLite devices, APC PDUs, AppAssure appliance, and ip KVM switch).

    I'll be curious to see what comes back when they re-quote.

    Why do they have to design a quote? You just tell them what you want, they give you a price. Other than "looking up the price", what are they doing?



Looks like your connection to MangoLassi was lost, please wait while we try to reconnect.