Unsolved Autoupdates Killed My Windows Server 2008 R2
-
@scottalanmiller said in Autoupdates Killed My Windows Server 2008 R2:
@Mike-Davis said in autoupdates killed my server:
I've had auto updates hose stuff before, but this one takes the cake. I took on a client with a Dell T310 server with Server 2008 R2 Standard on it. It's physical, not virtual.
You can sense the impending disaster right away.
That is why I always like to check backups before rebooting something that is unkown or not been rebooted in a while.
-
@Texkonc said in autoupdates killed my server:
That is why I always like to check backups before rebooting something that is unkown or not been rebooted in a while.
That's the odd thing. I do have backups and I was able to restore to a point 6 hours before the automatic updates went on. It isn't stuck in a boot loop, but I can't get to the desktop. It's possible that I haven't logged in to the desktop since the last auto update went on. I could try rolling back even further, but it would be better to do that on a test server.
-
@Texkonc said in Autoupdates Killed My Windows Server 2008 R2:
@dafyre said in autoupdates killed my server:
Only other thing I can think of would be to restore it to a hypervisor (and leave it disconnected) to see if that has any bearing on it... and then, you could restore older backups until you find one that works.
But with the old one running you will get conflicts. Power one off, power one on.
Delete the NIC so it doesnt preserve the MAC after the convert. new IP and rename it, then you can have both running.That's why I said leave it disconnected. So the VM doesn't try to take over... at least not right away!
-
Possibly failed update from before that are just sticking around and that is why the last company did not reboot the server? Maybe it is spiraled worse since you took over, but it was left over from the last guy.
-
@Mike-Davis does terminal server connection works?? what about windows offline updater http://download.wsusoffline.net/
-
Have you checked the CBS.log file in %windir%\logs\cbs
It will give more verbose errors usually. -
Update for all those that suggested ideas. I took @dafyre 's idea to restore it to a hyper visor. I went on site and I'm not sure why, but it took like 11 hours to copy the backups 1TB+ to an external USB drive. I brought that back to my office and started the restore. That took about 7 hours each time I did that. The first time I just restored the
drive. After messing around with bcdedit I still couldn't get the thing to boot. Veeam said that the M: drive was a system drive to, so I created another VM and restored the and M: drive. This time I could boot to the Dell system installer setup, but still couldn't boot the OS.Then I decided to restore to another physical Dell server I had on the bench. It booted no problem. Veeam boots you to Directory Services Restore mode and then you have to use msconfig to tell it to do a normal boot and you're good. I did that and it seemed like it was having the same issue where I logged in and it showed the desktop but wouldn't respond to the keyboard. The mouse moved, but wouldn't let me click. I just left it and came back an hour later. At that point it was fine. Not sure what the deal was.
There were a few variables to take in to consideration. Since the NIC was different, none of the network services came up. I also disabled a few things like CrashPlan because when the NIC does come online, I don't want it to try to backup to the cloud since it's a clone of the real server that is still on production.
At this point I'm going to try to P2V the thing.
-
NOT 100% Sure,
but if you go here:
C:\Windows\Installer
Can you try to sorted it by latest modified date, then run the MSI and see if some can be repaired on uninstalled ?
-
We patched over the weekend and had issues with server 2008 R2 on a .net update
It eventually went through but it took 4-5 hours on that one update
-
So I changed the disk layout of the VM I was trying to restore to and it booted like a champ. So now I have the offending server running as a VM. It seems as soon as I log in, CPU goes to 100% and it's all due to one svchost that calls Power, PlugPlay, and DcomLaunch. I can kill the svchost but then the server complains and wants to reboot.
-
I added more CPUs so that the offending process only took up 25% of the total system CPU. Then as I was going through the event viewer I noticed that many of the things that wouldn't start and were timing out had to do with the network. The NIC was still disconnected, so I enabled that. It wasn't recognized in device manager, so I installed the Hyper V integration tools. That fixed the NIC. I still had about 6 devices listed in device manager under "Other devices." Right clicking each one and telling it to update the driver fixed 4 out of 6.
-
@Mike-Davis said in Autoupdates Killed My Windows Server 2008 R2:
I added more CPUs so that the offending process only took up 25% of the total system CPU. Then as I was going through the event viewer I noticed that many of the things that wouldn't start and were timing out had to do with the network. The NIC was still disconnected, so I enabled that. It wasn't recognized in device manager, so I installed the Hyper V integration tools. That fixed the NIC. I still had about 6 devices listed in device manager under "Other devices." Right clicking each one and telling it to update the driver fixed 4 out of 6.
Did this improve the usability of the machine? Or was it still dog slow?
-
@dafyre said in Autoupdates Killed My Windows Server 2008 R2:
Did this improve the usability of the machine? Or was it still dog slow?
Yes, adding CPUs was the difference between being able to use it and it being dog slow.
-
I installed one automatic update, and now the system is blue screening. I knew there might be an issue since that's what started this whole thing, so I created a checkpoint before installing the update. I tried a couple things to clear the STOP: 0x0000006B blue screen that didn't work so I decided to revert to my checkpoint before the update.
I'm still getting the STOP: 0x0000006B and it keeps booting to the recovery option.
Building a new DC is looking better every minute. I can still log in to the old physical one I restored to DCpromo it out when I'm done.
-
@Mike-Davis said in Autoupdates Killed My Windows Server 2008 R2:
I installed one automatic update, and now the system is blue screening. I knew there might be an issue since that's what started this whole thing, so I created a checkpoint before installing the update. I tried a couple things to clear the STOP: 0x0000006B blue screen that didn't work so I decided to revert to my checkpoint before the update.
I'm still getting the STOP: 0x0000006B and it keeps booting to the recovery option.
Building a new DC is looking better every minute. I can still log in to the old physical one I restored to DCpromo it out when I'm done.
Will it work enough to replicate the AD stuff to the new server?
-
@dafyre said in Autoupdates Killed My Windows Server 2008 R2:
Will it work enough to replicate the AD stuff to the new server?
yes, Yesterday I restored one backup to a physical server and restored one back up to a VM. In the end, I had both working, but decided to move ahead with the virtual one. I can now power on the physical one, create a new VM, join the domain, etc and move forward with that. Of course I say this and most of what I have been working on for the last week should have worked...
-
@Mike-Davis said in Autoupdates Killed My Windows Server 2008 R2:
most of what I have been working on for the last week should have worked...
When dealing with computers, should work does not always equal does work, lol.
