Kooler on DFS-R Issues
-
DFS-R gets discussed a lot as a distributed high availability system for file servers and there are some regular issues with it that are often overlooked. @kooler did a big write up on this yesterday and I felt it would be good to replicate it here so that there is a reference point for it rather than just being a comment buried in a thread....
DFS-R "kinda works" but it has numerous drawbacks.
1) Non-transparent failover
Using Microsoft DFS-R for Failover
http://help.globalscape.com/help/wafs3/using_microsoft_dfs_for_failover.htm
In DFS failover, clients attempt to access another target in a referral after one of the targets fails to respond or is no longer part of the namespace. Clients must access a domain-based namespace by using the format \DomainName\RootName. If a client accesses a domain-based namespace directly on the root server (\RootServer\RootName), root target failover does not occur. DFS failover is only performed when a client opens a file or folder. If a client has files or folders open and attempts to read or write to them when the target server is unavailable, the application will receive a failure on that operation.
=> There's not much you can do here as DFS-R in kind of a maintenance mode, MSFT isn't releasing any updates for it.
2) Inability to replicate open files
DFS Replication: Frequently Asked Questions (FAQ)
https://technet.microsoft.com/en-us/library/cc773238(v=ws.10).aspx
Does DFS Replication replicate files that are being used by another application?
If an application opens a file and creates a file lock on it (preventing it from being used by other applications while it is open), DFS Replication will not replicate the file until it is closed. If the application opens the file with read-share access, the file can still be replicated.
Understanding (the Lack of) Distributed File Locking in DFSR
https://blogs.technet.microsoft.com/askds/2009/02/20/understanding-the-lack-of-distributed-file-lock...
Since users can modify data on multiple servers, and since each Windows server only knows about a file lock on itself, and since DFSR doesn’t know anything about those locks on other servers, it becomes possible for users to overwrite each other’s changes. DFSR uses a “last writer wins” conflict algorithm, so someone has to lose and the person to save last gets to keep their changes. The losing file copy is chucked into the ConflictAndDeleted folder.
=> This is huge issue actually, say Hyper-V, SQL Server, Exchange, Veeam etc are out of game as they either never close their files or close them after very big amount of data copied to them, this means you can't manage your connections reliably, it will be "all-or-nothing" sync channel usage at best (Veeam), or inability to work at all (Hyper-V & SQL).
You can leverage this by using third-party software like say PeerLock but it's expensive and not very popular. There are rumors MSFT had locking-aware DFS-R replication but they never released it to public (see my "maintenance mode" comment for 1).
3) PITA to resolve manual conflicts [after different people will edit files at their locations]
Understanding DFSR conflict algorithms (and doing something about conflicts)
https://blogs.technet.microsoft.com/askds/2010/01/05/understanding-dfsr-conflict-algorithms-and-doin...
That’s all well and good. But how do I get my conflicted files back when the “wrong” one wins?
You have a few options here:
Use DPM – Data Protection Manager provides on-the-fly backups of files and near-line recovery. This way your odds are highest that the latest versions of the file have been backed up.Use Volume Shadow Copies – You can configure automatic backups of files on your DFSR servers. Then when users delete or conflict files, the data can be easily restored. With a little training, your users can even restore files themselves and not have to spend time with the help desk. Note also that if you are still running XP or (Dog forbid) Win2000, you need to install a client to let users restore their own files. See TechNet and Windows Help for configuring this on a per-OS basis and make sure you read through the best practices info. VSC does not replace regular backups!Use backups – Windows Server Backup, NT Backup (if still on Win2003 R2), or 3rd parties should be used to back up
your data every day. This way no matter what, you can always get back to yesterday’s copy of a file.Use the restoredfsr.vbs script – Unsupported, as-is, and provided without warranty, this script may be your only hope if you have no created backups and shadow copies. Use it at your own risk. The script is hosted on Code Gallery (http://code.msdn.microsoft.com/restoredfsr). As always, the script requires you to edit a few variables before running – see the script for how-to documentation. You run it with:CSCRIPT.EXE restoredfsr.vbs=> You kind of can live with it if you don't do frequent changes.
So... Keeping in mind you can use free Hyper-V server and free StarWind virtual SAN to build a two-node shared nothing SMB3 clustered file server free of charge... I think it's time to retire DFS-R
See Step-by-Step guide:Hyper-V: Free “Shared Nothing” SMB3 Failover File Server
https://www.starwindsoftware.com/blog/part-2-smb-3-0-file-server-on-free-microsoft-hyper-v-server-20...
-
So... Keeping in mind you can use free Hyper-V server and free StarWind virtual SAN to build a two-node shared nothing SMB3 clustered file server free of charge... I think it's time to retire DFS-R See Step-by-Step guide:
Hyper-V: Free “Shared Nothing” SMB3 Failover File Server
https://www.starwindsoftware.com/blog/part-2-smb-3-0-file-server-on-free-microsoft-hyper-v-server-20...
Except this violates the Hyper-V Server 20xx license and is illegal. Do it on Windows Server and all is well. You'll need two Windows Server licenses, but StarWind vSAN is free. Or use Linux with StarWind vSAN.
I'm not too sure about SMB 3.x on Linux, but there may be ways.
-
As for the DFS-R...
I agree, it does have it's drawbacks and doesn't work well or as expected in some environments. But that doesn't mean it's not a good fit and won't work in others. I've seen it's success a lot.If you do have it set up and see that isn't not working as expected...
You can still use DFS, and for the replication part, StarWind vSAN would provide that part.I do like the Namespace idea and not being server/IP dependent like regular \server\share locations are. But yeah, it can be good fit some places, and not be a good fit in others.
@scottalanmiller said in Kooler on DFS-R Issues:
If a client accesses a domain-based namespace directly on the root server (\RootServer\RootName), root target failover does not occur.
I wouldn't tell users the names of the servers their data is on if using DFS. There's absolutely no reason to. All they need to know is the namespace: \domain\namespace
Storing your DFS shares on a StarWind vSAN SMB3 failover file server is awesome.
-
@Tim_G said in Kooler on DFS-R Issues:
As for the DFS-R...
I agree, it does have it's drawbacks and doesn't work well or as expected in some environments. But that doesn't mean it's not a good fit and won't work in others. I've seen it's success a lot.Years ago when I deployed this for our 2 locations (namespace and all) I pondered over the weekend what a genius move it was.
It only took a few weeks for the disillusionment to set in. I can't think of any scenario when DFRS would be good for anything else. It would have taken 9 years for our data to seed (if those servers were in the same room on the same switch. I even read in forums where admins where start DFS-R deployments and had allotted 6 months for the initial data to seed. Are you kidding? And just because it was "Microsoft" this was legit to them.
Even as I tried to use it for a single folder it was useless. RCOPY was far superior with a little scripting.
In my mind DFS-R is only good for the purpose I imagine it was originally created for, Active Directory replication...
-
@bigbear said in Kooler on DFS-R Issues:
@Tim_G said in Kooler on DFS-R Issues:
As for the DFS-R...
I agree, it does have it's drawbacks and doesn't work well or as expected in some environments. But that doesn't mean it's not a good fit and won't work in others. I've seen it's success a lot.Years ago when I deployed this for our 2 locations (namespace and all) I pondered over the weekend what a genius move it was.
It only took a few weeks for the disillusionment to set in. I can't think of any scenario when DFRS would be good for anything else. It would have taken 9 years for our data to seed (if those servers were in the same room on the same switch. I even read in forums where admins where start DFS-R deployments and had allotted 6 months for the initial data to seed. Are you kidding? And just because it was "Microsoft" this was legit to them.
Even as I tried to use it for a single folder it was useless. RCOPY was far superior with a little scripting.
In my mind DFS-R is only good for the purpose I imagine it was originally created for, Active Directory replication...
9 years? How many PB or EB were you initially replicating with DFS-R?
You would "seed" data with an external USB3 drive to the destination server, if you get less than 120 MBps.
But I've done terabytes over a weekend on numerous occasions, without "seeding", just using regular DFS-R from serverA to serverB. (millions of files)
What was the issue with your setup? Was it a ton of data over congested 10Mb Ethernet, or slow WAN?
-
The issue was the Microsoft product. This was Windows Server 2003 circa 2005 so it may pre-date your experience with Microsoft I don't know whats improved since then but it cant be much as you never hear anyone bragging about there awesome DFS-R deployment, lol.
As of Windows Server 2003 you couldn't seed a DFS-R partner with data, it had to be deployed at the first link and the server had to replicate it it build the library.
If you LOST that replication partner you had to start all over. In what version of Windows server did any of this change or get better?
-
@bigbear said in Kooler on DFS-R Issues:
The issue was the Microsoft product. This was Windows Server 2003 circa 2005 so it may pre-date your experience with Microsoft I don't know whats improved since then but it cant be much as you never hear anyone bragging about there awesome DFS-R deployment, lol.
As of Windows Server 2003 you couldn't seed a DFS-R partner with data, it had to be deployed at the first link and the server had to replicate it it build the library.
If you LOST that replication partner you had to start all over. In what version of Windows server did any of this change or get better?
Oh I see.
Must have been made better in 2008 or 2008 R2. I haven't used DFS-R before that (2008 R2). In my experience, it's always worked after configuring it properly. I use it on 2008 R2, 2012 R2, and 2016.
-
@Tim_G that is interesting to know, will do some reading up.
I nearly deployed a branch cache environment circa 2009 or 2010 but just didnt have the desire to go through trial and error. Have you had any experience with that?
-
@bigbear said in Kooler on DFS-R Issues:
@Tim_G that is interesting to know, will do some reading up.
I nearly deployed a branch cache environment circa 2009 or 2010 but just didnt have the desire to go through trial and error. Have you had any experience with that?
I wouldn't use it pre-2012 R2. But as far as implementing it, what's the occasion and requirement?
-
Few TB of live data being accessed between two sites WAS the scenario. The company has reorganized and split in to two new companies at this point. Now I am just curious as I have been in Microsoft mode all weekend.
Before it was a lot of CAD drawings access remotely, edited and collaborated on between offices.
-
@Tim_G said in Kooler on DFS-R Issues:
So... Keeping in mind you can use free Hyper-V server and free StarWind virtual SAN to build a two-node shared nothing SMB3 clustered file server free of charge... I think it's time to retire DFS-R See Step-by-Step guide:
Hyper-V: Free “Shared Nothing” SMB3 Failover File Server
https://www.starwindsoftware.com/blog/part-2-smb-3-0-file-server-on-free-microsoft-hyper-v-server-20...
Except this violates the Hyper-V Server 20xx license and is illegal. Do it on Windows Server and all is well. You'll need two Windows Server licenses, but StarWind vSAN is free. Or use Linux with StarWind vSAN.
I'm not too sure about SMB 3.x on Linux, but there may be ways.
I only skimmed things, was this enabling a role on the hyper-v server itself? If so, this is completely against the license agreement and not something anyone related to this forum should be supporting or posting.
-
@JaredBusch I actually couldn't figure out why the OP violated the licensing agreement. Its running a 3rd party image on the free version of Hyper-V
-
@JaredBusch said in Kooler on DFS-R Issues:
@Tim_G said in Kooler on DFS-R Issues:
So... Keeping in mind you can use free Hyper-V server and free StarWind virtual SAN to build a two-node shared nothing SMB3 clustered file server free of charge... I think it's time to retire DFS-R See Step-by-Step guide:
Hyper-V: Free “Shared Nothing” SMB3 Failover File Server
https://www.starwindsoftware.com/blog/part-2-smb-3-0-file-server-on-free-microsoft-hyper-v-server-20...
Except this violates the Hyper-V Server 20xx license and is illegal. Do it on Windows Server and all is well. You'll need two Windows Server licenses, but StarWind vSAN is free. Or use Linux with StarWind vSAN.
I'm not too sure about SMB 3.x on Linux, but there may be ways.
I only skimmed things, was this enabling a role on the hyper-v server itself? If so, this is completely against the license agreement and not something anyone related to this forum should be supporting or posting.
Not against the license, but not free. Requires consuming one of your two VM licenses to do it if it is the only workload.
-
@bigbear said in Kooler on DFS-R Issues:
@JaredBusch I actually couldn't figure out why the OP violated the licensing agreement. Its running a 3rd party image on the free version of Hyper-V
I spoke to them about this and apparently even MS has some uncertainty as to their license here and they've been trying to work it out. Some people at MS have told them that if they is used purely in the support of Hyper-V (including other Hyper-V) that it qualifies the same as a backup agent under the free use. but if you use it as a general SAN, of course it does not apply, like connecting it to ESXi. But because you are able to run pieces like this in a hyperconverged mode under the free license, it makes sense that you can in a non-hyperconverged mode as well.
-
But of course, if you are doing this to make an SMB server to serve files directly to end users, doesn't work without consuming a license.
-
@scottalanmiller said in Kooler on DFS-R Issues:
@bigbear said in Kooler on DFS-R Issues:
@JaredBusch I actually couldn't figure out why the OP violated the licensing agreement. Its running a 3rd party image on the free version of Hyper-V
I spoke to them about this and apparently even MS has some uncertainty as to their license here and they've been trying to work it out. Some people at MS have told them that if they is used purely in the support of Hyper-V (including other Hyper-V) that it qualifies the same as a backup agent under the free use. but if you use it as a general SAN, of course it does not apply, like connecting it to ESXi. But because you are able to run pieces like this in a hyperconverged mode under the free license, it makes sense that you can in a non-hyperconverged mode as well.
The physical server running Hyper-V Server (the hypervisor) cannot act as a file server, serving files to users or clients. It can only be used for supporting Hyper-V... including clustering, monitoring, etc.
I think I linked licensing information specific to this either here or on SW. I don't feel like digging it up atm, but will later if I need to.
-
@Tim_G said in Kooler on DFS-R Issues:
The physical server running Hyper-V Server (the hypervisor) cannot act as a file server, serving files to users or clients. It can only be used for supporting Hyper-V... including clustering, monitoring, etc.
The problem is, those are conflicting statements. Supporting Hyper-V Clustering is specifically what it is used for. Otherwise, you consume a license using Starwind always or even not using Starwind, just using local disks. But we know that local disks are okay. So using Starwind for Hyper-V clustering is logically okay as well. It just makes sense. It follows by the wording and the intent of the license.
Now maybe there is an argument that Hyper-V cannot provide its own storage via SMB3 and only iSCSI, in which case, I could see that being convoluted and weird, but could make sense.
-
@scottalanmiller said in Kooler on DFS-R Issues:
@Tim_G said in Kooler on DFS-R Issues:
The physical server running Hyper-V Server (the hypervisor) cannot act as a file server, serving files to users or clients. It can only be used for supporting Hyper-V... including clustering, monitoring, etc.
The problem is, those are conflicting statements. Supporting Hyper-V Clustering is specifically what it is used for. Otherwise, you consume a license using Starwind always or even not using Starwind, just using local disks. But we know that local disks are okay. So using Starwind for Hyper-V clustering is logically okay as well. It just makes sense. It follows by the wording and the intent of the license.
Now maybe there is an argument that Hyper-V cannot provide its own storage via SMB3 and only iSCSI, in which case, I could see that being convoluted and weird, but could make sense.
I meant clustering as in you can add your Hyper-V Server hypervisor to a hyper-v cluster. Then sure you can have a licensed windows VM running on that cluster that is doing the storage services and file serving... but not at the host level for the file services roles and features.
-
You can't install a 3rd party software to do storage or file server roles in place of the built in, thinking that is a way around it. It's not.
-
@Tim_G said in Kooler on DFS-R Issues:
... but not at the host level for the file services roles and features.
Right, not for FS role.
