Curious case of high VM disk usage upon RDP logout but stops upon RDP login.

  • At home I have a Promox host that I have a Win10 VM running Blue Iris (IP camera VMS). I've had it running for some time now (~year) with no issues. It's a simple setup. All my Proxmox VMs are hosted on a SSD. However, I do pass through two large spinning disks to the Blue Iris VM so that I can record video to the mechanical drives and not to the actually SSD that the VM is hosted on. Those two mechanical drives are only used by the Blue Iris VM. No other VM touches them.

    So yesterday I'm in the room where the host is located and I hear disk thrashing. You know, the sound of a hard drive being defragged? That kind of disk thrashing. Lots of reads and lots of writes. This is quite unusual as I've never heard those drives make noise ever. I open up my Proxmox GUI to see what is going on and I see that VM is showing a lot of of reads and writes. Proxmox is showing 50MB/s writes and 50MB/s reads all going at the same time. This is highly irregular. At any given time, Blue Iris is writing no more than 4MB/s (from my cameras to the drives) and reading virtually nothing so 50MB/s both ways is waaay of the charts. So I decide to quickly RDP into the VM and see what the hell is accessing the disk and as soon as I RDP in, the drives stop thrashing and disk access returns to normal levels.

    So I look around in task manager and everything looks fine and I chalk it up to nothing and log out of the RDP session. Well as soon as I log out, the drive thrashing fires right back up again. I finally figure out its tied to an RDP sessions after a couple times of logging in and out. Here is what Promox is showing for drive I/O for that VM with my annotations:

    Screenshot from 2020-09-08 07-47-47_modified.png

    You can see from the chart that whenever I'm logged out of RDP, disk I/O is real high. But as soon as I login, it dissapears and resumes normal I/O levels that I'm used to seeing which is about 3 - 4 MB/s write and little to no reads.

    How can I pin this down since whenever I log in, the task that is causing the issues pauses and nothing shows in task manager?

  • I came home this afternoon after lunch and noticed the server was back to whisper quiet. Checked the Proxmox I/O graphs and it back to its regular 3MB/s write and barely any reads. So I don't know what the hell Windows was doing for the last 24 hours but it seems to be done.

    I'm going to research this further because I have a feeling I'll see it again one day.

  • Have you checked to see if it was installing updates?

  • I am confused by the colors of the Graph, What does the teal color on the graph mean? It is not on the legend of the graph so I am trying to understand.

  • Windows defender scan or similar? Something that is designed to go low priority when it's being used and high priority when not. Many AV have that option but am uncertain what else within Windows will do that.

  • @dbeato Gold is reads and teal is writes. Writes were higher (slightly) than reads so you can't make out the gold reads very well.

    They use shading for those graphs. It would make more sense to make them transparent so you can see the line colors better.

  • @Danp @manxam Yeah upon login, there is nothing chewing disks or using CPU. Very strange...

    I wonder if there is something I can install that would log services running in the background and then I could look at it at a later time to determine what the service was that was using CPU, disk, etc...

  • @biggen : Something like procmon with thread profiling events enabled should give you an application's processor or disk usage over a period of time.

    EDIT: Good example here

Log in to reply