Caching Needs and SSDs



  • I saw this earlier today and see a recommendation to not run SSDs without a PERC card that has caching:
    https://community.spiceworks.com/topic/2005543-help-with-replacing-slow-server?source=homepage-feed.

    While RAID controllers often come with some cache (helpful for frequently accessed data), I thought the biggest benefit of getting a sizable cache on a RAID card was in conjunction with a battery on the RAID controller so you would not lose data if the system lost power while writing.

    How much of a performance impact can 1 GB or 2 GB of cache really have on a VM host with SSDs?


  • Service Provider

    @NetworkNerd said in Caching Needs and SSDs:

    How much of a performance impact can 1 GB or 2 GB of cache really have on a VM host with SSDs?

    A bit. For one thing, it dramatically increases the life of the SSDs, because it absorbs a huge percentage of the writes.


  • Service Provider

    One of the biggest benefits to Starwind is that it uses RAM cache in its SAN stack to give you millions of IOPS, instead of tens of thousands of IOPS, for lots of operations.


  • Service Provider

    @NetworkNerd said in Caching Needs and SSDs:

    While RAID controllers often come with some cache (helpful for frequently accessed data), I thought the biggest benefit of getting a sizable cache on a RAID card was in conjunction with a battery on the RAID controller so you would not lose data if the system lost power while writing.

    That would not make any sense since the need for the battery is only to protect the cache. Remove the cache and there is no purpose for the battery. The risk only comes from the cache.

    Also, the need for the battery went away long ago. That's a 2000's problem.


  • Service Provider

    @NetworkNerd said in Caching Needs and SSDs:

    I saw this earlier today and see a recommendation to not run SSDs without a PERC card that has caching:
    https://community.spiceworks.com/topic/2005543-help-with-replacing-slow-server

    That makes sense as you would not want to couple expensive high speed SSD with a cheap, slow RAID system. Bad use of funds.



  • @scottalanmiller said in Caching Needs and SSDs:

    @NetworkNerd said in Caching Needs and SSDs:

    I saw this earlier today and see a recommendation to not run SSDs without a PERC card that has caching:
    https://community.spiceworks.com/topic/2005543-help-with-replacing-slow-server

    That makes sense as you would not want to couple expensive high speed SSD with a cheap, slow RAID system. Bad use of funds.

    I know of a company that just bought a new Dell PowerEdge with local write-intensive SSDs to use for running an ERP system and SQL Server but only got the PERC H330. Oops for them. Of course, in their case, they may not notice compared to how much better things will be compared to their current hardware.


  • Service Provider

    @NetworkNerd said in Caching Needs and SSDs:

    I know of a company that just bought a new Dell PowerEdge with local write-intensive SSDs to use for running an ERP system and SQL Server but only got the PERC H330. Oops for them. Of course, in their case, they may not notice compared to how much better things will be compared to their current hardware.

    That's crazy. Unless they are doing software RAID. Then the cache is in system RAM and the battery is their UPS.

    Databases are specifically where you need gobs of RAM cache in front of your SSDs.



  • @scottalanmiller said in Caching Needs and SSDs:

    @NetworkNerd said in Caching Needs and SSDs:

    I know of a company that just bought a new Dell PowerEdge with local write-intensive SSDs to use for running an ERP system and SQL Server but only got the PERC H330. Oops for them. Of course, in their case, they may not notice compared to how much better things will be compared to their current hardware.

    That's crazy. Unless they are doing software RAID. Then the cache is in system RAM and the battery is their UPS.

    Databases are specifically where you need gobs of RAM cache in front of your SSDs.

    No software RAID will be used (hardware only) in this instance. It's not my environment but something I was told by a friend.


  • Service Provider

    @NetworkNerd said in Caching Needs and SSDs:

    @scottalanmiller said in Caching Needs and SSDs:

    @NetworkNerd said in Caching Needs and SSDs:

    I know of a company that just bought a new Dell PowerEdge with local write-intensive SSDs to use for running an ERP system and SQL Server but only got the PERC H330. Oops for them. Of course, in their case, they may not notice compared to how much better things will be compared to their current hardware.

    That's crazy. Unless they are doing software RAID. Then the cache is in system RAM and the battery is their UPS.

    Databases are specifically where you need gobs of RAM cache in front of your SSDs.

    No software RAID will be used (hardware only) in this instance. It's not my environment but something I was told by a friend.

    H330 doesn't do RAID. So they ARE doing software RAID. But they just aren't doing it properly.


  • Service Provider

    You were on the thread two years ago where we discussed how the H330 is not FakeRAID because Dell openly admits that it is software RAID and the card is just an HBA.

    https://community.spiceworks.com/topic/1038847-why-is-my-esxi-server-installation-not-detecting-raid-0-system-storage



  • @scottalanmiller said in Caching Needs and SSDs:

    You were on the thread two years ago where we discussed how the H330 is not FakeRAID because Dell openly admits that it is software RAID and the card is just an HBA.

    https://community.spiceworks.com/topic/1038847-why-is-my-esxi-server-installation-not-detecting-raid-0-system-storage

    Did you actually remember that thread off the top of your head or just find it after a little searching? I totally forgot about that thread.


  • Service Provider

    @NetworkNerd said in Caching Needs and SSDs:

    @scottalanmiller said in Caching Needs and SSDs:

    You were on the thread two years ago where we discussed how the H330 is not FakeRAID because Dell openly admits that it is software RAID and the card is just an HBA.

    https://community.spiceworks.com/topic/1038847-why-is-my-esxi-server-installation-not-detecting-raid-0-system-storage

    Did you actually remember that thread off the top of your head or just find it after a little searching? I totally forgot about that thread.

    I just Googled H330 and FakeRAID to see if we had posted about it because I was sure that it had been discussed. It came right up.


  • Service Provider

    Easy way to remember is that the H series are RAID with numbers like 6+ and are HBAs (non-RAID) with low numbers.



  • Double caching seems dangerous or at least slow things down a little.

    Do you mean like this:

    PERC Cache -> SSD Cache -> HDD Volume?

    I'd rather go: SSD Cache -> HDD Volume... using write-through to the SSD Cache.

    I can see going like this without issue:

    RAM Cache -> SSD Volume
    or
    RAM Cache -> HDD Volume
    or
    SSD Cache -> HDD Volume


  • Service Provider

    @Tim_G said in Caching Needs and SSDs:

    Double caching seems dangerous or at least slow things down a little.

    Not dangerous or slow. We do similar things for high speed websites. But in the opposite direction, of course.

    Double caching is no more dangerous than single caching, if caching is dangerous, it's dangerous. If it is safe, it doesn't matter how many times you do it.

    As for latency, it's not an issue and all hardware RAID systems like this do it under the hood because it's the only good way to add the SSD layer. You always want a RAM cache on the front end, and you need disks on the back end. The question is, do you want SSD in the middle or not.

    If you think about the performance leap from spinners to SSD, it makes total sense to have SSD in front of them for cache. Even a little SSD helps the spinners a LOT. Same difference from SSD to RAM. RAM is orders of magnitude faster than SSD, just like SSD is over spinners. So the gains are similar. And RAM doesn't wear like SSD does, so it doesn't just speed things up but allows for fewer and more efficient writes and that means longer SSD life which actually adds reliability.

    Also, cache allows writes to happen faster allowing more data to be protected before a failure.



  • @scottalanmiller said in Caching Needs and SSDs:

    @Tim_G said in Caching Needs and SSDs:

    Double caching seems dangerous or at least slow things down a little.

    Not dangerous or slow. We do similar things for high speed websites. But in the opposite direction, of course.

    Double caching is no more dangerous than single caching, if caching is dangerous, it's dangerous. If it is safe, it doesn't matter how many times you do it.

    As for latency, it's not an issue and all hardware RAID systems like this do it under the hood because it's the only good way to add the SSD layer. You always want a RAM cache on the front end, and you need disks on the back end. The question is, do you want SSD in the middle or not.

    If you think about the performance leap from spinners to SSD, it makes total sense to have SSD in front of them for cache. Even a little SSD helps the spinners a LOT. Same difference from SSD to RAM. RAM is orders of magnitude faster than SSD, just like SSD is over spinners. So the gains are similar. And RAM doesn't wear like SSD does, so it doesn't just speed things up but allows for fewer and more efficient writes and that means longer SSD life which actually adds reliability.

    Also, cache allows writes to happen faster allowing more data to be protected before a failure.

    I have a main Hypervisor running with a PERC H730p (2gb cache), configured with write-back (uses the 2gb cache) for all volumes, even the SSD SanDisk DAS Cache slow HDD volume.

    But if someone has a H330 (without a cache), you can still use an SSD cache (such as with SanDisk DAS Cache) perfectly fine. You don't NEED to have the onboard RAID card cache. You can even write-through to your SSD cache volume.


  • Service Provider

    @Tim_G said in Caching Needs and SSDs:

    But if someone has a H330 (without a cache), you can still use an SSD cache (such as with SanDisk DAS Cache) perfectly fine. You don't NEED to have the onboard RAID card cache. You can even write-through to your SSD cache volume.

    Of course you can use software to create caching, but you are doing everything in software, the H330 is just a SAS controller, nothing more. You can do RAM cache too. But your SSDs aren't getting caching like they should in the SanDisk DAS Cache scenario, so you aren't getting good performance and are unnecessarily beating up the SSDs.



  • I WISH there was a built-in way in Windows Server to use RAM as cache. I think it's awesome that StarWind has it.


  • Service Provider

    @Tim_G said in Caching Needs and SSDs:

    I WISH there was a built-in way in Windows Server to use RAM as cache. I think it's awesome that StarWind has it.

    Yeah, the reason it is not native, I would think, is because you need it lower in the stack. Putting it in a VM would be weird if individual VMs were doing loads on uncoordinated RAM cache syncs to partner VMs.


  • Service Provider

    @scottalanmiller said in Caching Needs and SSDs:

    Also, the need for the battery went away long ago. That's a 2000's problem.

    Do what? This was a standard feature as recently as 2012 on the Dell T600 line.


  • Service Provider

    @JaredBusch said in Caching Needs and SSDs:

    @scottalanmiller said in Caching Needs and SSDs:

    Also, the need for the battery went away long ago. That's a 2000's problem.

    Do what? This was a standard feature as recently as 2012 on the Dell T600 line.

    Standard as a cheaper downgrade, right?



  • @scottalanmiller said in Caching Needs and SSDs:

    @Tim_G said in Caching Needs and SSDs:

    But if someone has a H330 (without a cache), you can still use an SSD cache (such as with SanDisk DAS Cache) perfectly fine. You don't NEED to have the onboard RAID card cache. You can even write-through to your SSD cache volume.

    Of course you can use software to create caching, but you are doing everything in software, the H330 is just a SAS controller, nothing more. You can do RAM cache too. But your SSDs aren't getting caching like they should in the SanDisk DAS Cache scenario, so you aren't getting good performance and are unnecessarily beating up the SSDs.

    Good performance? You mean best performance. You're still getting much better performance than just having HDDs. Not everyone can afford or has a need for BEST performance. You still get good performance without a cache-cache. The whole point of the SSD cache is to speed things up. Without a cache on the PERC card in front of your SSD cache, you're not missing out on a ton of performance or life expectancy generally. Of course if your first goal is performance, then yes you'd get the H730p and not the H330.

    You can have a really nice performing system with a H330, SSD cache, and HDD storage volume, that costs a lot less than your "best" performing system.

    Enterprise SSDs have a crazy long life and you aren't beating them up at all on a system setup like that... if it was used for some really crazy throughput, it would look a lot different. Even using them as a cache (in normal use cases), they will still most likely out-live their life expectancy, or be replaced through your dell warranty if they do go early.

    I've seen them used with and without a cache in front of SSDs, with very little difference.

    Don't get me wrong, I do realize that an SSD used without a cache for it in an environment that has a lot of IOPS will perform less in performance and life expectancy. But under general use scenarios where a H330 would be used... no, not so much.


  • Service Provider

    @Tim_G said in Caching Needs and SSDs:

    Good performance? You mean best performance. You're still getting much better performance than just having HDDs.

    If you consider HDDs a base line. But I don't, I consider SSD a baseline today. And by good performance, I mean reasonable performance for the investment. Just like straight HDDs are pathetically slow without cache by our modern standards, so are SSDs.


  • Service Provider

    @Tim_G said in Caching Needs and SSDs:

    You can have a really nice performing system with a H330, SSD cache, and HDD storage volume, that costs a lot less than your "best" performing system.

    Why do you feel it will cost less? And more importantly, why do you feel it will be cost effective? And it's not JUST performance, it's also killing the SSDs.


  • Service Provider

    @Tim_G said in Caching Needs and SSDs:

    Enterprise SSDs have a crazy long life and you aren't beating them up at all on a system setup like that... if it was used for some really crazy throughput, it would look a lot different. Even using them as a cache (in normal use cases), they will still most likely out-live their life expectancy, or be replaced through your dell warranty if they do go early.

    True, but it is still a lot of increased risk for almost no cost savings.


  • Service Provider

    Things like databases feel the caching the most. Rapid small writes.



  • @scottalanmiller said in Caching Needs and SSDs:

    Things like databases feel the caching the most. Rapid small writes.

    That's why if you are running a database or other heavier IOP intensive applications, you wouldn't choose the H330. You'd put in some more money and do it right for the job.

    it's also killing the SSDs.

    It depends on the load, that's my point. I'm basing this on the fact that the engineer building the server is buying the right parts for the load, for the right costs.

    But a H730p is a lot more money than a H330. That costs more than replacing an SSD 8 years down the road... which would have most likely needed replacement even with caching.


  • Service Provider

    @Tim_G said in Caching Needs and SSDs:

    But a H730p is a lot more money than a H330. That costs more than replacing an SSD 8 years down the road... which would have most likely needed replacement even with caching.

    Haven't looked, what what is the price difference? It's bigger than just cache, though, one is hardware RAID and one is software, as well.


  • Service Provider

    Just looked on xByte, the price delta is $230. And that's for the big 2GB cache model.


  • Service Provider

    @scottalanmiller said in Caching Needs and SSDs:

    @JaredBusch said in Caching Needs and SSDs:

    @scottalanmiller said in Caching Needs and SSDs:

    Also, the need for the battery went away long ago. That's a 2000's problem.

    Do what? This was a standard feature as recently as 2012 on the Dell T600 line.

    Standard as a cheaper downgrade, right?

    Standard as in it was the "Dell Recommended" option at the time.

    And it looks like it was 2011.
    0_1497466472283_92c0bfeb-658e-4ae6-b1d7-e6ebb7e75689-image.png

    And it looks like the R720xd I purchased from XByte to replace that server has the PERC H710P with a battery.
    0_1497468089484_2ba4883f-421d-444d-8cee-175821fc74de-image.png
    0_1497468192102_bab37c26-a00b-4e68-b122-e4bbbe094e14-image.png



Looks like your connection to MangoLassi was lost, please wait while we try to reconnect.