@DustinB3403

I have made sure all the VMs have at least 20 gig of free vHDD space to eliminate any issue with drive space. We also have used a Fluke MicroScanner to check out our physical cables and found a couple of suspect cables that we replaced as well.

Here are some test results, SMB file copy from a 2008R2 VM that has NIC offloading turned on to the new 2012R2 box with no PV tools so its using the base Windows Realtek driver and got 24 Mbps. Not great but much better than the 64-72 Kbps I get between two 2008R2 boxes with offloading turned on. Same test from 2008R2 VM with NIC offloading turned off and I get the same 24 Mbps. All of these VMs are on the same host.

Ok installed the PV tools on the 2012R2 server. Copy from a 2008R2 VM with NIC Offloading ON and the 350 meg file copied in a flash. Resource Monitor showed 2 Gbps. Copy from the VM with NIC Offloading OFF and the copied at 225 Mbps.

Maybe I should just start do inplace upgrades on my servers!!!!

@scottalanmiller

I was going to message via SpiceWorks but I thought this may be a better place per your prior message to me.

I am still have some issues with XenServer and I don't think Citrix pointed me in the right direction. I finally nailed down that I had an issue with the PV NIC when transferring data via SMB shares. I was able to show that it was a Server 2008R2 related issue as my W7 VMs were just fine so the Host, Physical NIC, and Switch were all preforming well with the W7 VM and not the 2008R2 VMs. Citrix had me disable NIC Offloading. I did that Friday and SMB data rates shot thru the roof. Cool right? Well today one of my software packages that uses SMB and SQL calls from the client, a 2008R2 RDS server, would not preform at any usable rate. Worked with the software support folks and would up turning NIC Offloading back on on the application/SQL server and the RDS server and bang the software works ok on the SQL calls but I am back to slow (64 KB) data rates via SMB. So now I have no clue what to do to try to fix this mess.

Host IO is good at 250-300 MBps and all VMs are on local storage. Using XenServer 6.5SP1010. Any Ideas?

@DustinB3403

I have made sure all the VMs have at least 20 gig of free vHDD space to eliminate any issue with drive space. We also have used a Fluke MicroScanner to check out our physical cables and found a couple of suspect cables that we replaced as well.

Here are some test results, SMB file copy from a 2008R2 VM that has NIC offloading turned on to the new 2012R2 box with no PV tools so its using the base Windows Realtek driver and got 24 Mbps. Not great but much better than the 64-72 Kbps I get between two 2008R2 boxes with offloading turned on. Same test from 2008R2 VM with NIC offloading turned off and I get the same 24 Mbps. All of these VMs are on the same host.

Ok installed the PV tools on the 2012R2 server. Copy from a 2008R2 VM with NIC Offloading ON and the 350 meg file copied in a flash. Resource Monitor showed 2 Gbps. Copy from the VM with NIC Offloading OFF and the copied at 225 Mbps.

Maybe I should just start do inplace upgrades on my servers!!!!

I can't prove the performance issue is tied to Windows updates. What I can prove is how the VMs are preforming today and relate that back to the fact that 6-12 months ago I didn't seem to have these issues with the same software packages. Performance started going down gradually as we didn't really notice issues until it "suddenly" occurred to us that things were not running well. In the overnight I have spun up a 2012R2 VM to do some testing with. I am also wondering if I need to flash the BIOS on the underlying hardware as if the software layer was updated and is expecting the NIC to perform a specific way but the BIOS isn't doing that then maybe my issue is there?

Yes with the VM that would not copy at a reasonable rate. That is why I moved to 6.5SP1 from 6.2 base. It seemed like everything was ok when we first installed but over time (I am guessing with windows updates) it just kept getting slower and slower. I traced thru everything I could think of from an IO perspective moved the hosts from RAID 6 to RAID 10, on the VMs upped the vCPU counts, and upped the RAM. When none of that worked I purchased Citrix support and they came up with the NIC offloading.

Yes the XenTools are up to date with the latest and greatest after all the host updates were applied. Based on my testing I am thinking that my issue is specific to Server 2008R2 and its interaction with the PV NIC but I was hoping to get some direction on how to figure that out specifically. The VM I had previously been having issues with was taking 6-11 hours to copy 1.5 GB from my IBM Mainframe SMB share to itself and then processing that into a SQL database. After turning off NIC Offloading that VM has run that process in less than 10 min. The software package I am have trouble with now also use SMB and SQL but is sending data to the clients via SQL. When watching Processor Monitor you can see SMB crank up as part of the system process and get north of 500 meg but SQL never gets above 50kbps. Same VMs turn Offloading back on and SQL cranks up north of 100 meg but SMB never gets over 70kbps.

I am nearing a point where I am about ready to run AD Prep and move the forest up to a 2012R2 functional level and then run an in place upgrade on one or more of these servers and see if that helps at all.

@scottalanmiller

I was going to message via SpiceWorks but I thought this may be a better place per your prior message to me.

I am still have some issues with XenServer and I don't think Citrix pointed me in the right direction. I finally nailed down that I had an issue with the PV NIC when transferring data via SMB shares. I was able to show that it was a Server 2008R2 related issue as my W7 VMs were just fine so the Host, Physical NIC, and Switch were all preforming well with the W7 VM and not the 2008R2 VMs. Citrix had me disable NIC Offloading. I did that Friday and SMB data rates shot thru the roof. Cool right? Well today one of my software packages that uses SMB and SQL calls from the client, a 2008R2 RDS server, would not preform at any usable rate. Worked with the software support folks and would up turning NIC Offloading back on on the application/SQL server and the RDS server and bang the software works ok on the SQL calls but I am back to slow (64 KB) data rates via SMB. So now I have no clue what to do to try to fix this mess.

Host IO is good at 250-300 MBps and all VMs are on local storage. Using XenServer 6.5SP1010. Any Ideas?