So I ended up making a special trip because it was bothering the crap out of me. I brought a different usb drive with me to test. The replacement usb drive had the exact same problem. I plugged both drives into my laptop and both worked fine. Finally, I reformatted the drives from my laptop and then reconnected to the server and everything worked perfectly. I shutdown the server a couple times and did a couple cold boots just to make sure everything stuck and all was well. The original formatting of the drives were all NTFS so I am not sure what difference reformatting them from my laptop made but seemed to do the trick.
I appreciate all the feedback from everyone.