Handling DNS in a Single Active Directory Domain Controller Environment
-
@travisdh1 said in Handling DNS in a Single Active Directory Domain Controller Environment:
@dafyre said in Handling DNS in a Single Active Directory Domain Controller Environment:
@jaredbusch said in Handling DNS in a Single Active Directory Domain Controller Environment:
What really needs to be laid out here is a list of what needs done on both sides, both proactively and reactively after a failure. At that point relative costs can be estimated.
I certainly what @JaredBusch mentions would be a good grounding point for this point of discussion... Let's first describe the business scenario.
Company Details for Scenario 1
Acme, Inc.
24 Employees
1 x Virtualization Host
1 x AD Server (AD, DNS, DHCP) VM
Y x other VMs
Email is hosted on O365.
(we don't care about other VMs for sake of this discussion, do we?)
1 x Network RouterAssumptions:
- All devices use the router for DNS1, and AD Server for DNS2.
- Router points to AD Server for DNS1, and CloudFlare for DNS2.
- Company already owns a working backup product
Scenario 1:
Problem: AD Server VM Blows up, Blue Screens, Gets Deleted or just won't boot.
Impact: Services Requiring AD for authentication will not work. Devices that were working when the AD Server died continue working until DHCP lease time runs out. Internet is up since the router can use CloudFlare for DNS.
Solution: Restore VM from most recent backup into new VM on the Virtualization host.
Cost Formula: Hours Downtime * Lost Productivity (if Any) = Total Cost
**Cost: 2 hrs * $5000/hr = $10,000Does that oversimplify the discussion or provide enough details?
Getting there. What service broke because AD was down? Most of the time, AD could be down and nobody would know the difference. To have cost associated with AD being down, a service that doesn't cache credentials has to be authenticating with it.
Seriously, try it in your home lab sometime. Just shut down any AD servers you have running and see how long it takes for something to break.
First thing that comes to mind: NextCloud with AD integration, RocketChat with AD integration.
For the case of my scenario, we don't worry about WHAT broke. If you look closely at my Cost Formula... It was Lost Productivity (if Any)... because you're right, just because AD is down, doesn't necessarily mean the entire business just stops.
-
@dafyre said in Handling DNS in a Single Active Directory Domain Controller Environment:
@jaredbusch said in Handling DNS in a Single Active Directory Domain Controller Environment:
What really needs to be laid out here is a list of what needs done on both sides, both proactively and reactively after a failure. At that point relative costs can be estimated.
I certainly what @JaredBusch mentions would be a good grounding point for this point of discussion... Let's first describe the business scenario.
Company Details for Scenario 1
Acme, Inc.
24 Employees
1 x Virtualization Host
1 x AD Server (AD, DNS, DHCP) VM
Y x other VMs
Email is hosted on O365.
(we don't care about other VMs for sake of this discussion, do we?)
1 x Network RouterAssumptions:
- All devices use the router for DNS1, and AD Server for DNS2.
- Router points to AD Server for DNS1, and CloudFlare for DNS2.
- Company already owns a working backup product
Scenario 1:
Problem: AD Server VM Blows up, Blue Screens, Gets Deleted or just won't boot.
Impact: Services Requiring AD for authentication will not work. Devices that were working when the AD Server died continue working until DHCP lease time runs out. Internet is up since the router can use CloudFlare for DNS.
Solution: Restore VM from most recent backup into new VM on the Virtualization host.
Cost Formula: Hours Downtime * Lost Productivity (if Any) = Total Cost
Cost: 2 hrs * $5000/hr = $10,000Does that oversimplify the discussion or provide enough details?
No, because it doesn't take 2 hours to restore a 40GB VM. It takes 5 minutes. If it happens over the weekend, and business takes place during the weekend, that's a different story. For many, it won't even matter and can be handled on Monday morning or VERY QUICKLY Sunday night. You don't need to be on-prem to restore a VM.
-
What about an SMB who already has the mitigations in place (everything is set up correctly) for a single-DC environment?
-
What about automation? What if AD cannot be reached, so a bunch of other automatic checks take place, and if determined, automatically restores the DC? This would be rather simple to set up.
-
@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:
What about an SMB who already has the mitigations in place (everything is set up correctly) for a single-DC environment?
@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:
What about automation? What if AD cannot be reached, so a bunch of other automatic checks take place, and if determined, automatically restores the DC? This would be rather simple to set up.
Not sure how this is even germane to the discussion. We are talking about best practices and recommendations for AD implementation. If everything has the additional investment that you're talking about then single DC AD would be best, but what you're describing is a ways down the decision tree. It might come in to consideration depending on the skill sets of the technicians and the investment the business wants to put into place. However what you're describing requires a higher skill level than most smaller SMBs would have access to, or significantly more investment than a second DC. All part of the cost/risk calculation, but it doesn't land in the auto recommend category, just like a redundant DC does not.
-
@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:
No, because it doesn't take 2 hours to restore a 40GB VM. It takes 5 minutes. If it happens over the weekend, and business takes place during the weekend, that's a different story. For many, it won't even matter and can be handled on Monday morning or VERY QUICKLY Sunday night. You don't need to be on-prem to restore a VM.
It might very well take two hours if you have cloud backup. Actually, you should probably be very glad if you can restore a tiny little 40GB VM from the cloud in two hours
But even if the backup is local you still have to determine what the problem is first. Why would the VM crash if there is not a hardware problem on the VM host? What does the disks on the host looks like, do we have bad sectors? Or is it a NIC problem on the VM host or a port on the switch? You can't determine what the problem is and also fix it in 5 minutes, that's completely unrealistic.
Also, if you're not on-prem and don't have a working AD, are you even able to remote in and access anything?
-
@pete-s said in Handling DNS in a Single Active Directory Domain Controller Environment:
@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:
No, because it doesn't take 2 hours to restore a 40GB VM. It takes 5 minutes. If it happens over the weekend, and business takes place during the weekend, that's a different story. For many, it won't even matter and can be handled on Monday morning or VERY QUICKLY Sunday night. You don't need to be on-prem to restore a VM.
It might very well take two hours if you have cloud backup. Actually, you should probably be very glad if you can restore a tiny little 40GB VM from the cloud in two hours
But even if the backup is local you still have to determine what the problem is first. Why would the VM crash if there is not a hardware problem on the VM host? What does the disks on the host looks like, do we have bad sectors? Or is it a NIC problem on the VM host or a port on the switch? You can't determine what the problem is and also fix it in 5 minutes, that's completely unrealistic.
Why not isolated the bad DC VM for troubleshooting later and restore the backup now?
-
@black3dynamite said in Handling DNS in a Single Active Directory Domain Controller Environment:
@pete-s said in Handling DNS in a Single Active Directory Domain Controller Environment:
@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:
No, because it doesn't take 2 hours to restore a 40GB VM. It takes 5 minutes. If it happens over the weekend, and business takes place during the weekend, that's a different story. For many, it won't even matter and can be handled on Monday morning or VERY QUICKLY Sunday night. You don't need to be on-prem to restore a VM.
It might very well take two hours if you have cloud backup. Actually, you should probably be very glad if you can restore a tiny little 40GB VM from the cloud in two hours
But even if the backup is local you still have to determine what the problem is first. Why would the VM crash if there is not a hardware problem on the VM host? What does the disks on the host looks like, do we have bad sectors? Or is it a NIC problem on the VM host or a port on the switch? You can't determine what the problem is and also fix it in 5 minutes, that's completely unrealistic.
Why not isolated the bad DC VM for troubleshooting later and restore the backup now?
If you fear the VM host has a severe disk or disc controller problem it doesn't make sense to keep it running. Then you'd want to shutdown all VMs and run diagnostics before taking it back up again.
-
@pete-s said in Handling DNS in a Single Active Directory Domain Controller Environment:
It might very well take two hours if you have cloud backup. Actually, you should probably be very glad if you can restore a tiny little 40GB VM from the cloud in two hours
Why would your only backups exist in the cloud over a slow connection? Mistake number 1.
@pete-s said in Handling DNS in a Single Active Directory Domain Controller Environment:
But even if the backup is local you still have to determine what the problem is first. Why would the VM crash if there is not a hardware problem on the VM host?
Because Windows? I don't know. I didn't come up with the scenario. They don't in my experience crash. Windows Updates maybe? Who knows. Lots of reasons a Windows VM could crash, lots of reasons a physical host or host OS could crash too.
@pete-s said in Handling DNS in a Single Active Directory Domain Controller Environment:
You can't determine what the problem is and also fix it in 5 minutes, that's completely unrealistic.
This is true regardless of whatever way you do things. Assuming it's the VM, and it's crashed. Restore it in 5 minutes from on-prem backups, or take the time to fix it in hours, cease fsmo roles, and rebuild a new DC from scratch in hours.
-
@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:
You can't determine what the problem is and also fix it in 5 minutes, that's completely unrealistic.
This is true regardless of whatever way you do things. Assuming it's the VM, and it's crashed. Restore it in 5 minutes from on-prem backups, or take the time to fix it in hours, cease fsmo roles, and rebuild a new DC from scratch in hours.
I agree. I was just saying if we were to calculate the cost of the downtime, the down time will not be 5 minutes. You have to calculate the time it takes for everything including the users having problems, to them calling you (and get a hold of you), time for troubleshooting and then to the last 5 minutes of restoring the VM. So 2 hours it was
-
@dafyre said in Handling DNS in a Single Active Directory Domain Controller Environment:
Assumptions:
- All devices use the router for DNS1, and AD Server for DNS2.
- Router points to AD Server for DNS1, and CloudFlare for DNS2.
- Company already owns a working backup product
Your DNS is off, otherwise this is a good layout.
Everything should always point to AD DNS first in an AD environment.So it should look like this.
Assumptions:
- All devices use the AD DC for DNS 1 and the router for DNS 2
- Router points to AD Server for DNS 1, and CloudFlare for DNS 2.
-
@pmoncho said in Handling DNS in a Single Active Directory Domain Controller Environment:
@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:
@pmoncho said in Handling DNS in a Single Active Directory Domain Controller Environment:
@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:
@kelly said in Handling DNS in a Single Active Directory Domain Controller Environment:
just challenging the "most commonly correct approach" statement
It seems you are mistaking the "most common approach" with the "most common correct approach". I haven't been around the SMB as much as JB, but I'm assuming the most common approach to SMB DC implementations are incorrect. Meaning, 2+ DCs are being used when 1 should be used. Perhaps two DCs are used because so many other things are done incorrectly, it's thought 1 should't be used due to so many other things not properly in place, but that's besides the point in my reply here.
IMHO, SMB's use 2 DC's (me included) because it is drilled over and over in our heads by outside forces, including the application developers and the OS companies themselves. On top of that, we are completely stupid if we don't have a second DC if the hardware is available. So to follow "Best Practices," SMB's just do it. It doesn't necessarily mean that things are done incorrectly though. It mostly means, we (aka I) have an extra DC there sitting, waiting, getting monthly updates and then gather more dust for years on end all in the name of protection and risk reduction.
That is why coming here and having extensive discussions about general topics has helped me changed my own thoughts about system/network design in SMB's.
Then I assume you have an extra everything if it costs less than $5k, correct? Especially if other things depend on it... such as redundant ISP, all redundant switches, definitely redundant LoB services, etc... if not, why choose only a DC over things that would be way more beneficial to have HA? If you have extra hardware, extra software, etc... that would go unused and be wasted otherwise, then sure, it could make more sense, but could still cause the same amount of benefits and negatives.
Just because a company has an extra DC doesn't mean every process/product/connection needs to be duplicated. If there are two hosts an extra DC is peanuts. No $5K is needed, $800 tops and there is value (reduced risk) in that $800. Plus, as been mentioned, ceasing roles is less time and MUCH less panic than restoring a VM.
Theres so much more though - now you have to make sure there are no replication issues, and you should likely be backing up that VM (it is a VM, right?) also. You could do it free, but assuming you're using a backup product, that might require another license because it's another box, so more costs. It's also additional time doing updates, 2 boxes vs 1.
-
@dafyre said in Handling DNS in a Single Active Directory Domain Controller Environment:
@jaredbusch said in Handling DNS in a Single Active Directory Domain Controller Environment:
What really needs to be laid out here is a list of what needs done on both sides, both proactively and reactively after a failure. At that point relative costs can be estimated.
I certainly what @JaredBusch mentions would be a good grounding point for this point of discussion... Let's first describe the business scenario.
Company Details for Scenario 1
Acme, Inc.
24 Employees
1 x Virtualization Host
1 x AD Server (AD, DNS, DHCP) VM
Y x other VMs
Email is hosted on O365.
(we don't care about other VMs for sake of this discussion, do we?)
1 x Network RouterAssumptions:
- All devices use the router for DNS1, and AD Server for DNS2.
- Router points to AD Server for DNS1, and CloudFlare for DNS2.
- Company already owns a working backup product
Scenario 1:
Problem: AD Server VM Blows up, Blue Screens, Gets Deleted or just won't boot.
Impact: Services Requiring AD for authentication will not work. Devices that were working when the AD Server died continue working until DHCP lease time runs out. Internet is up since the router can use CloudFlare for DNS.
Solution: Restore VM from most recent backup into new VM on the Virtualization host.
Cost Formula: Hours Downtime * Lost Productivity (if Any) = Total Cost
Cost: 2 hrs * $5000/hr = $10,000Does that oversimplify the discussion or provide enough details?
Ok - now the question is - how likely is that?
I thought we already covered that the AD DNS should be first - though I can see arguments on both sides - so, whatever. I'm guessing the AD DNS being first would actually be best from a performance POV because one less hope when looking for things when all things are working correctly.
-
@dashrender said in Handling DNS in a Single Active Directory Domain Controller Environment:
@dafyre said in Handling DNS in a Single Active Directory Domain Controller Environment:
@jaredbusch said in Handling DNS in a Single Active Directory Domain Controller Environment:
What really needs to be laid out here is a list of what needs done on both sides, both proactively and reactively after a failure. At that point relative costs can be estimated.
I certainly what @JaredBusch mentions would be a good grounding point for this point of discussion... Let's first describe the business scenario.
Company Details for Scenario 1
Acme, Inc.
24 Employees
1 x Virtualization Host
1 x AD Server (AD, DNS, DHCP) VM
Y x other VMs
Email is hosted on O365.
(we don't care about other VMs for sake of this discussion, do we?)
1 x Network RouterAssumptions:
- All devices use the router for DNS1, and AD Server for DNS2.
- Router points to AD Server for DNS1, and CloudFlare for DNS2.
- Company already owns a working backup product
Scenario 1:
Problem: AD Server VM Blows up, Blue Screens, Gets Deleted or just won't boot.
Impact: Services Requiring AD for authentication will not work. Devices that were working when the AD Server died continue working until DHCP lease time runs out. Internet is up since the router can use CloudFlare for DNS.
Solution: Restore VM from most recent backup into new VM on the Virtualization host.
Cost Formula: Hours Downtime * Lost Productivity (if Any) = Total Cost
Cost: 2 hrs * $5000/hr = $10,000Does that oversimplify the discussion or provide enough details?
Ok - now the question is - how likely is that?
I thought we already covered that the AD DNS should be first - though I can see arguments on both sides - so, whatever. I'm guessing the AD DNS being first would actually be best from a performance POV because one less hope when looking for things when all things are working correctly.
I'm still all for LANless.
At home, I log in to my home Windows computer with my Outlook.com account. That's basically the same as if you used AADDS for your SMB. Then you'd use your AAD login for everything else, and only use software that supports that.
-
But I must add you don't have to go MS to be LANless, above was just an example.
-
@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:
@dashrender said in Handling DNS in a Single Active Directory Domain Controller Environment:
@dafyre said in Handling DNS in a Single Active Directory Domain Controller Environment:
@jaredbusch said in Handling DNS in a Single Active Directory Domain Controller Environment:
What really needs to be laid out here is a list of what needs done on both sides, both proactively and reactively after a failure. At that point relative costs can be estimated.
I certainly what @JaredBusch mentions would be a good grounding point for this point of discussion... Let's first describe the business scenario.
Company Details for Scenario 1
Acme, Inc.
24 Employees
1 x Virtualization Host
1 x AD Server (AD, DNS, DHCP) VM
Y x other VMs
Email is hosted on O365.
(we don't care about other VMs for sake of this discussion, do we?)
1 x Network RouterAssumptions:
- All devices use the router for DNS1, and AD Server for DNS2.
- Router points to AD Server for DNS1, and CloudFlare for DNS2.
- Company already owns a working backup product
Scenario 1:
Problem: AD Server VM Blows up, Blue Screens, Gets Deleted or just won't boot.
Impact: Services Requiring AD for authentication will not work. Devices that were working when the AD Server died continue working until DHCP lease time runs out. Internet is up since the router can use CloudFlare for DNS.
Solution: Restore VM from most recent backup into new VM on the Virtualization host.
Cost Formula: Hours Downtime * Lost Productivity (if Any) = Total Cost
Cost: 2 hrs * $5000/hr = $10,000Does that oversimplify the discussion or provide enough details?
Ok - now the question is - how likely is that?
I thought we already covered that the AD DNS should be first - though I can see arguments on both sides - so, whatever. I'm guessing the AD DNS being first would actually be best from a performance POV because one less hope when looking for things when all things are working correctly.
I'm still all for LANless.
At home, I log in to my home Windows computer with my Outlook.com account. That's basically the same as if you used AADDS for your SMB. Then you'd use your AAD login for everything else, and only use software that supports that.
OK - but that's another conversation, not this one.
-
@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:
But I must add you don't have to go MS to be LANless, above was just an example.
LOL - A stand along Mac or CentOS box is LANLess.
-
@dashrender said in Handling DNS in a Single Active Directory Domain Controller Environment:
@pmoncho said in Handling DNS in a Single Active Directory Domain Controller Environment:
@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:
@pmoncho said in Handling DNS in a Single Active Directory Domain Controller Environment:
@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:
@kelly said in Handling DNS in a Single Active Directory Domain Controller Environment:
just challenging the "most commonly correct approach" statement
It seems you are mistaking the "most common approach" with the "most common correct approach". I haven't been around the SMB as much as JB, but I'm assuming the most common approach to SMB DC implementations are incorrect. Meaning, 2+ DCs are being used when 1 should be used. Perhaps two DCs are used because so many other things are done incorrectly, it's thought 1 should't be used due to so many other things not properly in place, but that's besides the point in my reply here.
IMHO, SMB's use 2 DC's (me included) because it is drilled over and over in our heads by outside forces, including the application developers and the OS companies themselves. On top of that, we are completely stupid if we don't have a second DC if the hardware is available. So to follow "Best Practices," SMB's just do it. It doesn't necessarily mean that things are done incorrectly though. It mostly means, we (aka I) have an extra DC there sitting, waiting, getting monthly updates and then gather more dust for years on end all in the name of protection and risk reduction.
That is why coming here and having extensive discussions about general topics has helped me changed my own thoughts about system/network design in SMB's.
Then I assume you have an extra everything if it costs less than $5k, correct? Especially if other things depend on it... such as redundant ISP, all redundant switches, definitely redundant LoB services, etc... if not, why choose only a DC over things that would be way more beneficial to have HA? If you have extra hardware, extra software, etc... that would go unused and be wasted otherwise, then sure, it could make more sense, but could still cause the same amount of benefits and negatives.
Just because a company has an extra DC doesn't mean every process/product/connection needs to be duplicated. If there are two hosts an extra DC is peanuts. No $5K is needed, $800 tops and there is value (reduced risk) in that $800. Plus, as been mentioned, ceasing roles is less time and MUCH less panic than restoring a VM.
Theres so much more though - now you have to make sure there are no replication issues, and you should likely be backing up that VM (it is a VM, right?) also. You could do it free, but assuming you're using a backup product, that might require another license because it's another box, so more costs. It's also additional time doing updates, 2 boxes vs 1.
In the scenario of 2 DC's, the VM would be backed up but is it worth it? Restoring a DC VM with multiple DC's has a higher probability of creating replication issues.
The backup product plus a server license for it, would not be included in the costs per this discussion as every scenario would have this cost (unless using windows backup but you still need somewhere to put the backup files).
As for updates, I view this as a HUGE value. Now, one can update the 2nd DC (aka non-FSMO role holder) first and if there is an issue, it doesn't effect any part of the network allowing the admin to NOT run updates on other servers.
If an SMB cannot afford a 2nd DC, then they definitely cannot afford a test environment. So all updates are run directly on production servers. We all know MS can really fork up and update or two.
My patch monthly patch process goes like this; On Sat of "Patch Tuesday" week, I update my 2nd DC and allow it to run till Tuesday. If no issues, I then proceed to other systems during the week or the next Sat. I have had 2 patch issues on a very very generic 2nd DC (Only, AD/DNS nothing else) over the years that could have cost big down time had it run on all production servers. IMHO, that safety, sanity, and security has a lot of value. Like the value investing axiom goes, "Price is what you pay, Value is what you get"
Paying a single OS license for YEARS of a production update server can have a value of 3X its worth.
I am not saying that a very small 10 person SMB shop with one host, 3 VM's (AD/DNS, FS, RDS) should have two DC's. But when you start creeping up to 40-50 users and maybe 100 remote clients, then maybe two DC's come in handy by reducing risk.
-
@obsolesce said in Handling DNS in a Single Active Directory Domain Controller Environment:
@dafyre said in Handling DNS in a Single Active Directory Domain Controller Environment:
@jaredbusch said in Handling DNS in a Single Active Directory Domain Controller Environment:
What really needs to be laid out here is a list of what needs done on both sides, both proactively and reactively after a failure. At that point relative costs can be estimated.
I certainly what @JaredBusch mentions would be a good grounding point for this point of discussion... Let's first describe the business scenario.
Company Details for Scenario 1
Acme, Inc.
24 Employees
1 x Virtualization Host
1 x AD Server (AD, DNS, DHCP) VM
Y x other VMs
Email is hosted on O365.
(we don't care about other VMs for sake of this discussion, do we?)
1 x Network RouterAssumptions:
- All devices use the router for DNS1, and AD Server for DNS2.
- Router points to AD Server for DNS1, and CloudFlare for DNS2.
- Company already owns a working backup product
Scenario 1:
Problem: AD Server VM Blows up, Blue Screens, Gets Deleted or just won't boot.
Impact: Services Requiring AD for authentication will not work. Devices that were working when the AD Server died continue working until DHCP lease time runs out. Internet is up since the router can use CloudFlare for DNS.
Solution: Restore VM from most recent backup into new VM on the Virtualization host.
Cost Formula: Hours Downtime * Lost Productivity (if Any) = Total Cost
Cost: 2 hrs * $5000/hr = $10,000Does that oversimplify the discussion or provide enough details?
No, because it doesn't take 2 hours to restore a 40GB VM. It takes 5 minutes. If it happens over the weekend, and business takes place during the weekend, that's a different story. For many, it won't even matter and can be handled on Monday morning or VERY QUICKLY Sunday night. You don't need to be on-prem to restore a VM.
We're talking an SMB here. While I generally think you're right, we need to have a clearly defined scenario. The SMB IT guy may be a good one and get it done in 5 minutes... Or he may have no idea what he's doing and it takes him 4 hours instead of two. No sense in nitpicking how long it takes to do the restore since I'm telling you how long it takes to do it.
-
@jaredbusch said in Handling DNS in a Single Active Directory Domain Controller Environment:
@dafyre said in Handling DNS in a Single Active Directory Domain Controller Environment:
Assumptions:
- All devices use the router for DNS1, and AD Server for DNS2.
- Router points to AD Server for DNS1, and CloudFlare for DNS2.
- Company already owns a working backup product
Your DNS is off, otherwise this is a good layout.
Everything should always point to AD DNS first in an AD environment.So it should look like this.
Assumptions:
- All devices use the AD DC for DNS 1 and the router for DNS 2
- Router points to AD Server for DNS 1, and CloudFlare for DNS 2.
Updated the scenario. Thanks for that.
