Windows Server 2003 Cluster Dead
-
Looks like the power went out on the node last night, which might have been a trigger.
-
Yesterday morning, we got a SCSI error that the RAID didn't respond in time.
-
The power down happened about 18 hours after the SCSI error.
-
Last Event Log from Node 2 was 7 hours after the SCSI event, 11 hours before the Node 1 power cycle.
-
13 hours AFTER the power cycle, the event log reports that the Quorum disk "Q" cannot be found.
-
Obviously once the Q disk was missing, it could not join the cluster.
-
Doing a controller power cycle now. Bringing down the physical cluster now, then the DAS. Then going to power on the DAS, give it time, and bring the nodes up. Expect very little, but it is a place to start.
-
DAS is powering up, lights on it do not look good.
-
10 drives in the array, believed to be RAID 10. 2 drives in RAID 1 as well.
-
One drive in the large array is flashing orange, so looks like one drive has failed.
All other drives are green.
-
Bringing up Node 1 again now. With only one drive failed in the DAS unit, any RAID (other than RAID 0) should have survived.
-
Okay, that process brought things up. Not the cluster, but the disks are back. We can see the Quorum plus other disks now.
-
Trying to bring up Node 2 now, but I'm not hopeful on that.
-
Node 1 is healthy, Node 2 is gone. Cluster won't come up, but the workloads did. So they are good for now.
-
Do they have a plan to replace this outdated tech with something current?
-
@Danp said in Windows Server 2003 Cluster Dead:
Do they have a plan to replace this outdated tech with something current?
Yes, there was a six month plan in place already, but it just got moved to something like a six day plan.
-
@scottalanmiller said in Windows Server 2003 Cluster Dead:
@Danp said in Windows Server 2003 Cluster Dead:
Do they have a plan to replace this outdated tech with something current?
Yes, there was a six month plan in place already, but it just got moved to something like a six day plan.
Love it when that happens!!
-
@FATeknollogee said in Windows Server 2003 Cluster Dead:
@scottalanmiller said in Windows Server 2003 Cluster Dead:
@Danp said in Windows Server 2003 Cluster Dead:
Do they have a plan to replace this outdated tech with something current?
Yes, there was a six month plan in place already, but it just got moved to something like a six day plan.
Love it when that happens!!
The system is still f***** because they have to replace it today and they have to worry about good backups today.
2003 is ancient
-
I'm guessing the thing hasn't been maintained at all which would have brought this about sooner but in a controlled manner.
-
@DustinB3403 said in Windows Server 2003 Cluster Dead:
@FATeknollogee said in Windows Server 2003 Cluster Dead:
@scottalanmiller said in Windows Server 2003 Cluster Dead:
@Danp said in Windows Server 2003 Cluster Dead:
Do they have a plan to replace this outdated tech with something current?
Yes, there was a six month plan in place already, but it just got moved to something like a six day plan.
Love it when that happens!!
The system is still f***** because they have to replace it today and they have to worry about good backups today.
2003 is ancient
Backups are running today.
