System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7
-
06:00:01 AM all 0.64 0.00 0.14 0.02 0.43 98.76 06:10:01 AM all 1.57 0.00 0.29 0.06 0.62 97.47 06:20:01 AM all 0.85 0.00 0.21 0.03 0.56 98.35 06:30:01 AM all 1.34 0.00 0.22 0.07 0.83 97.53 06:40:01 AM all 0.79 0.23 0.30 0.10 0.61 97.97 06:50:01 AM all 1.15 0.00 0.19 0.06 0.62 97.98 07:00:01 AM all 0.67 0.00 0.15 0.03 0.51 98.65 07:10:02 AM all 1.17 0.00 0.19 0.07 0.62 97.95 07:20:02 AM all 0.71 0.00 0.16 0.03 0.36 98.74 07:30:01 AM all 1.36 0.00 0.22 0.07 0.55 97.81 07:40:01 AM all 5.82 0.00 0.26 0.03 0.63 93.25 07:50:01 AM all 15.07 0.00 0.35 0.07 1.14 83.36 08:00:01 AM all 8.80 0.00 0.27 0.04 0.79 90.11 08:10:01 AM all 16.20 0.00 0.36 0.07 1.13 82.24 08:20:01 AM all 10.55 0.00 0.24 0.04 1.02 88.15 08:30:01 AM all 16.24 0.00 0.32 0.10 1.20 82.15 08:40:01 AM all 10.00 0.00 0.30 0.03 0.96 88.72 08:50:02 AM all 16.61 0.00 0.30 0.08 1.19 81.82 09:00:01 AM all 11.60 0.00 0.27 0.16 1.06 86.91 09:10:01 AM all 17.84 0.00 0.30 0.09 1.24 80.54 09:10:01 AM CPU %user %nice %system %iowait %steal %idle 09:20:01 AM all 11.77 0.00 0.27 0.13 0.90 86.93 09:30:01 AM all 17.04 0.00 0.29 0.09 1.16 81.41 09:40:01 AM all 13.90 0.00 0.31 0.06 1.14 84.59 09:50:02 AM all 23.21 0.00 0.42 0.08 1.70 74.60 10:00:01 AM all 12.95 0.00 0.27 0.06 1.15 85.56 Average: all 4.19 0.00 0.23 0.06 0.70 94.81No one is currently using SC, so no connections
-
-
Where is the DB stored?
-
htop @ 16:10
sar @ 16:00
-
@hobbit666 said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
Where is the DB stored?
/opt/screenconnect/App_Data
-
@hobbit666 That's what our top output looked like before this issue hit.
-
@hobbit666
How many clients (devices) do you have? We have hit 540 as of this week. -
Also, what version of SC are you running?
Display Current Installed Version from puTTy
echo $(cat /opt/[Ss]creen[Cc]onnect/Bin/*ScreenConnect.Core.dll | LC_ALL=C tr -d -c "[:print:]" | grep -E -o "ProductVersion[[:digit:]]+(.[[:digit:]]+)+")
(Code block makes text invisable)
-
Ran an Audit, which shows system connection history information.
Pegged the system, and memory jumped to 1.3GB
After the Audit finished, it didn't release the memory
-
@gjacobse So it's a memory leak?
-
@gjacobse What command ran the audit?
-
@wirestyle22 said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@gjacobse So it's a memory leak?
No reason to think that. Used memory and memory leak are very different things. Also, the box is normal once his process is done. A leak keeps using memory more and more over time.
-
-
-
12:00:01 PM all 13.72 0.00 0.56 0.06 1.59 84.06 12:10:01 PM all 19.92 0.00 0.60 0.09 1.77 77.62 12:20:01 PM all 11.98 0.00 0.48 0.10 1.02 86.42 12:30:02 PM all 17.21 0.00 0.45 0.10 1.17 81.08 12:40:01 PM all 12.69 0.00 0.43 0.04 1.13 85.71 12:50:01 PM all 17.98 0.00 0.47 0.08 1.41 80.05 01:00:01 PM all 13.49 0.00 0.46 0.06 1.17 84.82 01:10:01 PM all 19.12 0.00 0.55 0.10 1.50 78.73 01:20:01 PM all 11.94 0.00 0.48 0.05 0.98 86.55 01:30:02 PM all 19.56 0.00 0.56 0.09 1.48 78.31 01:40:01 PM all 24.13 0.00 0.74 0.09 1.21 73.83 01:50:01 PM all 26.44 0.00 0.65 0.08 1.34 71.48 02:00:01 PM all 17.44 0.00 0.54 0.04 1.08 80.90 02:10:01 PM all 28.65 0.00 0.80 0.08 1.80 68.67 02:20:01 PM all 11.86 0.00 0.45 0.03 0.95 86.71 02:30:01 PM all 17.95 0.00 0.52 0.07 1.27 80.19 02:40:01 PM all 11.69 0.00 0.44 0.04 0.94 86.90 02:50:02 PM all 27.31 0.00 0.67 0.06 1.48 70.49 03:00:01 PM all 15.61 0.00 0.49 0.03 1.16 82.70 03:10:01 PM all 17.35 0.00 0.49 0.07 1.21 80.88 03:20:01 PM all 12.84 0.00 0.54 0.03 1.05 85.54 03:30:01 PM all 26.24 0.00 0.82 0.07 1.83 71.03 03:40:02 PM all 13.70 0.00 0.60 0.03 1.41 84.25 03:50:01 PM all 20.65 0.00 0.62 0.07 1.49 77.18 04:00:01 PM all 11.31 0.00 0.49 0.03 1.10 87.06 04:10:01 PM all 20.25 0.00 0.64 0.07 1.44 77.61 04:20:01 PM all 10.64 0.00 0.45 0.03 0.83 88.05 04:30:01 PM all 20.96 0.00 0.60 0.07 1.37 77.00 04:40:01 PM all 5.77 0.00 0.53 0.03 1.02 92.65 04:50:02 PM all 8.36 0.00 0.44 0.06 0.91 90.23 05:00:01 PM all 1.78 0.00 0.34 0.03 0.50 97.36 05:10:01 PM all 7.63 0.00 0.47 0.06 0.84 91.00 Average: all 9.29 0.00 0.36 0.06 0.98 89.30 -
I don't think it shows in the last few screen shots,.. but found a PID what was running for 4hr 54m,.. Talking to @scottalanmiller came up on killing that process to see if it would release memory and it didn't.
But, in that, I found another similar process that had been running for more than 10 hours, so I killed that one.
Memory is now down to 325M.
-
Tag 1 is the pid that seemed to have been the lynch pin
-
So what is routinely kicking off that problem child, though?
-
Sorted by Time:
-
Starting to look for differences since the DB Maint should have run about an hour ago.
]$ cd /opt/screenconnect//App_Data/ [ App_Data]$ ls -l total 101304 -rw-r--r-- 1 root root 454 Aug 25 19:15 ExtensionConfiguration.xml drwxr-xr-x 2 root root 4096 Nov 8 09:44 Helper -rw-r--r-- 1 root root 654 Aug 25 19:15 License.xml -rw-r--r-- 1 root root 23699 Dec 16 11:26 Role.xml -rw-r--r-- 1 root root 25210880 Dec 17 01:59 Session.db -rw-r--r-- 1 root root 32768 Dec 17 02:04 Session.db-shm -rw-r--r-- 1 root root 78403632 Dec 17 02:04 Session.db-wal -rw-r--r-- 1 root root 4030 Oct 25 08:55 SessionEventTrigger.xml -rw-r--r-- 1 root root 5178 Dec 2 08:13 SessionGroup.xml drwxr-xr-x 14 root root 4096 Dec 12 12:55 Toolbox -rw-r--r-- 1 root root 0 Dec 14 17:24 User.cd60043d-556e-4b23-9ef1-9959e6cea952.xml -rw-r--r-- 1 root root 21908 Dec 16 17:24 User.xml
