Server will not shut down
-
Given the setup, my inkling is that you have an issue with the iSCSI Initiator. The Microsoft iSCSI Initiator is known to be flaky and can easily be causing an issue. Both architecturally and practically due to the use of Windows it is standard practice to present SAN storage as a local drive via VMware to the VM rather than inside of the VM. VMware's handling of iSCSI is much more robust than Windows is, and you need only one connection per physical host rather than one per VM.
-
@JaredBusch said in Server will not shut down:
Why 4 vCPU each? That seems excessive.
Agreed. We don't expect the software to use that much, but we're building it to the specs that our vendor has requested.
-
@dafyre said in Server will not shut down:
@JaredBusch said in Server will not shut down:
Why 4 vCPU each? That seems excessive.
Agreed. We don't expect the software to use that much, but we're building it to the specs that our vendor has requested.
With virtualization that can cause performance problems. The vendor may be injecting a mistake here. Having more cores than are needed will degrade performance. And... can cause lockups in rare cases.
-
@scottalanmiller said in Server will not shut down:
@dafyre said in Server will not shut down:
No, we don't have anything running on them yet. The iSCSI initiator service is the only thing that has been set up, and our usual disable IEC, and UAC type stuff.
Otherwise, it is a basic Server 2012 R2 install.
Why use iSCSI from inside of the VM?
Because it works that way?
That's the way I would set it up. Although this time, I'm actually not the one who set up the master image. We're connecting to a SAN to store large amounts of video for security cameras.
-
@dafyre said in Server will not shut down:
Why use iSCSI from inside of the VM?
Because it works that way?
Well it doesn't work well that way and in this case, it's possible that it doesn't work at all
-
@dafyre said in Server will not shut down:
That's the way I would set it up. Although this time, I'm actually not the one who set up the master image. We're connecting to a SAN to store large amounts of video for security cameras.
I'm not questioning the SAN or the iSCSI. I'm questioning the configuration which is extremely complex and fragile compared to a more elegant, more modern configuration using VMware as designed.
-
@scottalanmiller said in Server will not shut down:
Given the setup, my inkling is that you have an issue with the iSCSI Initiator. The Microsoft iSCSI Initiator is known to be flaky and can easily be causing an issue. Both architecturally and practically due to the use of Windows it is standard practice to present SAN storage as a local drive via VMware to the VM rather than inside of the VM. VMware's handling of iSCSI is much more robust than Windows is, and you need only one connection per physical host rather than one per VM.
I've set up shared storage with VMware before, and IIRC, isn't there a limit on the size of the disk image that you can set up? Our VMware disk images would be ~20 TB each.
-
@dafyre said in Server will not shut down:
I've set up shared storage with VMware before, and IIRC, isn't there a limit on the size of the disk image that you can set up? Our VMware disk images would be ~20 TB each.
62TB
-
@scottalanmiller said in Server will not shut down:
@dafyre said in Server will not shut down:
I've set up shared storage with VMware before, and IIRC, isn't there a limit on the size of the disk image that you can set up? Our VMware disk images would be ~20 TB each.
62TB
Cool. I'll pass that info along and see if he wants to set it up that way.
-
@dafyre said in Server will not shut down:
@scottalanmiller said in Server will not shut down:
@dafyre said in Server will not shut down:
I've set up shared storage with VMware before, and IIRC, isn't there a limit on the size of the disk image that you can set up? Our VMware disk images would be ~20 TB each.
62TB
Cool. I'll pass that info along and see if he wants to set it up that way.
Assuming you are not on some ancient VMware. It's been that big for a while.
I would take a "moment" and make one VM or two that use the "direct to VMware" SAN method and test that and see if it fixes things. It simplifies your setup by quite a bit and would be better for the long term. It's more efficient to the SAN, so might be very good for your performance, too. And avoiding that Windows component is always worth it. Just makes things simpler and more abstracted - leveraging virtualization for what it is built for.
-
It might not be the issue, but it's a place to start.
-
@scottalanmiller said in Server will not shut down:
It might not be the issue, but it's a place to start.
Seems legit, lol.
I've done set ups using the iSCSI initiator before (I have two of them running here now -- Failover Clusters) and never had any problems at all. So something in this setup is the issue.
-
@dafyre said in Server will not shut down:
@scottalanmiller said in Server will not shut down:
It might not be the issue, but it's a place to start.
Seems legit, lol.
I've done set ups using the iSCSI initiator before (I have two of them running here now -- Failover Clusters) and never had any problems at all. So something in this setup is the issue.
It has to work "most of the time" or they couldn't even include it. But like a lot of things in the Windows storage ecosystem (which includes Dynamic Disks, DFS and Software RAID), the iSCSI components are weak and fragile and can cause lots of issues. This is why companies like @StarWind_Software have taken the time to rewrite and replace those components for enterprise use because the built in ones just aren't up to the task. And now that Windows is "always" run as a VM and the standard pattern is using the SAN to the hypervisor, Microsoft has no reason to fix or improve what is included any longer. So now it is stagnating, as well.
I've never tried Windows with iSCSI setup and cloned. I'm wondering if cloning iSCSI is part of the issue.
-
@scottalanmiller said in Server will not shut down:
@dafyre said in Server will not shut down:
@scottalanmiller said in Server will not shut down:
It might not be the issue, but it's a place to start.
Seems legit, lol.
I've done set ups using the iSCSI initiator before (I have two of them running here now -- Failover Clusters) and never had any problems at all. So something in this setup is the issue.
It has to work "most of the time" or they couldn't even include it. But like a lot of things in the Windows storage ecosystem (which includes Dynamic Disks, DFS and Software RAID), the iSCSI components are weak and fragile and can cause lots of issues. This is why companies like @StarWind_Software have taken the time to rewrite and replace those components for enterprise use because the built in ones just aren't up to the task. And now that Windows is "always" run as a VM and the standard pattern is using the SAN to the hypervisor, Microsoft has no reason to fix or improve what is included any longer. So now it is stagnating, as well.
I've never tried Windows with iSCSI setup and cloned. I'm wondering if cloning iSCSI is part of the issue.
I would use it if it worked badly. But it does work, and in my experience, when it works, it works well... Or it works so badly that it's not even usable.
We're actually talking about the iSCSI stuff being in the clone now. We'll see what boss says when we all get back from lunch.
-
@dafyre said in Server will not shut down:
I would use it if it worked badly. But it does work, and in my experience, when it works, it works well... Or it works so badly that it's not even usable.
That could be what's happening here
-
Yeah i was wondering if the cloned ISCSI stuff was the problem as well.
Does each VM get it's own LUN on the SAN?
-
@scottalanmiller said in Server will not shut down:
@dafyre said in Server will not shut down:
I would use it if it worked badly. But it does work, and in my experience, when it works, it works well... Or it works so badly that it's not even usable.
That could be what's happening here
I'd expect nothing less from Microsoft. However, it would seem that our master image works flawlessly... So we'll still blame Microsoft, lol.
-
@dafyre said in Server will not shut down:
@scottalanmiller said in Server will not shut down:
@dafyre said in Server will not shut down:
I would use it if it worked badly. But it does work, and in my experience, when it works, it works well... Or it works so badly that it's not even usable.
That could be what's happening here
I'd expect nothing less from Microsoft. However, it would seem that our master image works flawlessly... So we'll still blame Microsoft, lol.
I would expect the master to work, but i was wondering, did you sysprep the master, then run through the mini config and it's still working?
If yes, I wonder if there is some sort of hardware ID associated with ISCSI that needs to be unique to each machine, and if the master is online, then the clones won't work? just a thought.
-
@Dashrender said in Server will not shut down:
@dafyre said in Server will not shut down:
@scottalanmiller said in Server will not shut down:
@dafyre said in Server will not shut down:
I would use it if it worked badly. But it does work, and in my experience, when it works, it works well... Or it works so badly that it's not even usable.
That could be what's happening here
I'd expect nothing less from Microsoft. However, it would seem that our master image works flawlessly... So we'll still blame Microsoft, lol.
I would expect the master to work, but i was wondering, did you sysprep the master, then run through the mini config and it's still working?
If yes, I wonder if there is some sort of hardware ID associated with ISCSI that needs to be unique to each machine, and if the master is online, then the clones won't work? just a thought.
Yeah. The master has been offline the whole time while we are tinkering. I thought you may be onto something for a minute, but alas, the IQN (initiator name) does change with the system after it has been cloned & sysprepped.
I just tested it... it changes with the hostname if you don't set one yourself.
-
Well, it turns out the fix was much easier than I anticipated. Not 100% certain, but we have at least one VM that is now cooperating with us, whereas before we had none.
It's one of those I feel stupid things... Both NICs are on the same subnet. So all we had to do was manually set the Metric for each interface, and then everything magically started working.
