Looking for some insight/input for setting up a "hot site"/fail-over environment
-
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
We have a few locations that are all relatively close by
That is probably the craziest thing about going through all this hassle and being in the same geographic region. I would seriously consider a colo outside of the area. I mean you do all that work and put in all these expenses and you have an area wide disaster and its potentially gone.
Colocation costs are not much more than hosting at your own facility. -
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
Could replicate data at the storage controller level:The SCv2020 does have embedded REPL ports specifically for replicating to another SCv2020 via iSCSI - this requires more additional licensing which is fine
Does this maintain anything other than crash consistency? Unless the SCv2020 is just automating VMware to do the work, this isn't a failover.
-
@IRJ said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
We have a few locations that are all relatively close by
That is probably the craziest thing about going through all this hassle and being in the same geographic region. I would seriously consider a colo outside of the area. I mean you do all that work and put in all these expenses and you have an area wide disaster and its potentially gone.
Colocation costs are not much more than hosting at your own facility.Sometimes, it's even less! Not often, but it can happen.
-
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
I think I might go ahead and use Veeam's VM replication for a warm site setup.
Way safer, and probably cheaper.
-
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@wrx7m said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@wrx7m said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
What type of Windows licensing do you have? Replication can be affected by that.
We have Microsoft Volume Licensing and I haven't looked into weather or not we'd have to purchase "double" or not.. I'm not sure but I thought a backup environment doesn't quite count as production.
I would say "Hot"=Active.
hot/warm/standby environment is more what I'm talking about. I do plan to look into the licensing requirements involved. We have Datacenter for the vmware environment.
Just an FYI / PSA... if you use Scale HC3, this kind of replication and failover is all included in the base purchase.
-
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@wrx7m said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
What type of Windows licensing do you have? Replication can be affected by that.
We have Microsoft Volume Licensing and I haven't looked into weather or not we'd have to purchase "double" or not.. I'm not sure but I thought a backup environment doesn't quite count as production.
You need the second site licensed if it is hot, meaning turned on. You do not need anything if it is cold, meaning powered off. Powered off, it's just a backup that's extremely "ready to go".
-
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
I think the SCv2020 is EoL so I'm not sure if I would buy a new one or upgrade/buy double or if the SCv2020 can replicate to anything else
Typically replication requires same to same, but not absolutely always.
-
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
Could replicate virtual machines at the VMware level but we will need a new vSphere license, setup may be more complex (also fine)
Replication always has to be at this level. You can't safely replicate below it. Storage doesn't have the ability to replicate on its own. If your VMware licensing doesn't allow for replication, that answers all of the questions about the storage lower down the stack. Even if the SC has the ability to replicate, it depends on this feature of VMware to execute it correctly. If you do it without VMware doing the heavy lifting you get inconsistent data. Often looks good in tests, often corrupt in a real life failover.
-
@IRJ said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
We have a few locations that are all relatively close by
That is probably the craziest thing about going through all this hassle and being in the same geographic region. I would seriously consider a colo outside of the area. I mean you do all that work and put in all these expenses and you have an area wide disaster and its potentially gone.
Colocation costs are not much more than hosting at your own facility.yeah that would be preferred but we likely won't be able to get our company owners to spend that much money. Setting up a minimal warm site in our other building would be much cheaper/easier.
-
@scottalanmiller said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@wrx7m said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
What type of Windows licensing do you have? Replication can be affected by that.
We have Microsoft Volume Licensing and I haven't looked into weather or not we'd have to purchase "double" or not.. I'm not sure but I thought a backup environment doesn't quite count as production.
You need the second site licensed if it is hot, meaning turned on. You do not need anything if it is cold, meaning powered off. Powered off, it's just a backup that's extremely "ready to go".
ok that's good to know. I don't doubt you, but do you know where this info is explicitly stated by MS? I'd like to have it on hand. I may just have to dig through the licensing ToS or something.
-
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@wrx7m said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@wrx7m said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
What type of Windows licensing do you have? Replication can be affected by that.
We have Microsoft Volume Licensing and I haven't looked into weather or not we'd have to purchase "double" or not.. I'm not sure but I thought a backup environment doesn't quite count as production.
I would say "Hot"=Active.
hot/warm/standby environment is more what I'm talking about. I do plan to look into the licensing requirements involved. We have Datacenter for the vmware environment.
I think only cold gets away with no licensing.
-
How are you planning on handling the IP change - I'm assuming each building is in a different IP range, so routing will become an issue - just another piece of the puzzle.
-
@Dashrender said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
How are you planning on handling the IP change - I'm assuming each building is in a different IP range, so routing will become an issue - just another piece of the puzzle.
Stop building environments with static IPs period. That is an old way of thinking
-
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@IRJ said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
We have a few locations that are all relatively close by
That is probably the craziest thing about going through all this hassle and being in the same geographic region. I would seriously consider a colo outside of the area. I mean you do all that work and put in all these expenses and you have an area wide disaster and its potentially gone.
Colocation costs are not much more than hosting at your own facility.yeah that would be preferred but we likely won't be able to get our company owners to spend that much money. Setting up a minimal warm site in our other building would be much cheaper/easier.
I went through what you're going through about 10 years ago. Almost identical.
What we found is that everything we put in place and all the procedures we had to follow to meet our BCDR was so much work that we decided it was extremely stupid to spend hundreds of thousands of dollars in licensing, manpower, and procedure to get our stuff operational to meet our BCDR and then fail because of a location.
My boss asked for a remote site instead of using another local facility and it was a no brainer to everyone involved include C level non IT people. They understood costs and effort level to get there.
-
@IRJ said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@Dashrender said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
How are you planning on handling the IP change - I'm assuming each building is in a different IP range, so routing will become an issue - just another piece of the puzzle.
Stop building environments with static IPs period. That is an old way of thinking
yeah, OK - with IPv6, that's not really a thing anyway, right?
(not that IPv6 has anything to do with anything regarding this project - it's just the future way of IPing and brings along lack of static IPs)
-
@Dashrender said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@IRJ said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@Dashrender said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
How are you planning on handling the IP change - I'm assuming each building is in a different IP range, so routing will become an issue - just another piece of the puzzle.
Stop building environments with static IPs period. That is an old way of thinking
yeah, OK - with IPv6, that's not really a thing anyway, right?
(not that IPv6 has anything to do with anything regarding this project - it's just the future way of IPing and brings along lack of static IPs)
I'm not even thinking about IPv6 but yeah having a DNS service that can do health checks is ideal and route to another host when when you spin it up.
This can all be automated using terraform in the cloud and even to alot of extent on prem. For example you could spin up hosts on prem and pull all the ip addresses from your terraform state file for easy import into your DNS server.
On AWS, as soon as terraform creates the instance you build your DNS entry. The old one fails health check and it points to new server. Boom you are done
-
@IRJ said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@Dashrender said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
How are you planning on handling the IP change - I'm assuming each building is in a different IP range, so routing will become an issue - just another piece of the puzzle.
Stop building environments with static IPs period. That is an old way of thinking
Nobody said anything about static IPs at all. I assume he just meant in terms of having a secondary environment (especially a hot one) there would be an additional network involved to support the DR site - but the servers and appliances being replicated/sitting on stand-by would be on the same subnet/have the same IP addresses as the primary environment - which could lead to conflicts/be an obstacle.
-
@scottalanmiller said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@wrx7m said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
What type of Windows licensing do you have? Replication can be affected by that.
We have Microsoft Volume Licensing and I haven't looked into weather or not we'd have to purchase "double" or not.. I'm not sure but I thought a backup environment doesn't quite count as production.
You need the second site licensed if it is hot, meaning turned on. You do not need anything if it is cold, meaning powered off. Powered off, it's just a backup that's extremely "ready to go".
I found the info I needed in the Windows Server 2016 Licensing Guide and if I understand it correctly, we would need Software Assurance coverage with our licensing, which we do not have. We purchased licenses only (with necessary CALs).
-
@Dashrender said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@wrx7m said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@dave247 said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@wrx7m said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
What type of Windows licensing do you have? Replication can be affected by that.
We have Microsoft Volume Licensing and I haven't looked into weather or not we'd have to purchase "double" or not.. I'm not sure but I thought a backup environment doesn't quite count as production.
I would say "Hot"=Active.
hot/warm/standby environment is more what I'm talking about. I do plan to look into the licensing requirements involved. We have Datacenter for the vmware environment.
I think only cold gets away with no licensing.
That's correct.
-
@IRJ said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@Dashrender said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@IRJ said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
@Dashrender said in Looking for some insight/input for setting up a "hot site"/fail-over environment:
How are you planning on handling the IP change - I'm assuming each building is in a different IP range, so routing will become an issue - just another piece of the puzzle.
Stop building environments with static IPs period. That is an old way of thinking
yeah, OK - with IPv6, that's not really a thing anyway, right?
(not that IPv6 has anything to do with anything regarding this project - it's just the future way of IPing and brings along lack of static IPs)
I'm not even thinking about IPv6 but yeah having a DNS service that can do health checks is ideal and route to another host when when you spin it up.
This can all be automated using terraform in the cloud and even to alot of extent on prem. For example you could spin up hosts on prem and pull all the ip addresses from your terraform state file for easy import into your DNS server.
On AWS, as soon as terraform creates the instance you build your DNS entry. The old one fails health check and it points to new server. Boom you are done
Right. Using things like Consul for service discovery makes it easy.
Also even just doing automated configs for your load balancers works as well.
