System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7
-
]$ ls -l total 29724 -rw-r--r-- 1 root root 454 Aug 25 19:15 ExtensionConfiguration.xml drwxr-xr-x 2 root root 4096 Nov 8 09:44 Helper -rw-r--r-- 1 root root 654 Aug 25 19:15 License.xml -rw-r--r-- 1 root root 23699 Dec 16 11:26 Role.xml -rw-r--r-- 1 root root 1232896 Dec 19 20:14 Session.db -rw-r--r-- 1 root root 24956928 Dec 19 20:11 Session.db.bak -rw-r--r-- 1 root root 32768 Dec 19 20:18 Session.db-shm -rw-r--r-- 1 root root 4128272 Dec 19 20:18 Session.db-wal -rw-r--r-- 1 root root 4030 Oct 25 08:55 SessionEventTrigger.xml -rw-r--r-- 1 root root 5178 Dec 2 08:13 SessionGroup.xml drwxr-xr-x 14 root root 4096 Dec 12 12:55 Toolbox -rw-r--r-- 1 root root 0 Dec 14 17:24 User.cd60043d-556e-4b23-9ef1-9959e6cea952.xml -rw-r--r-- 1 root root 21908 Dec 19 20:13 User.xml -
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
-rw-r--r-- 1 root root 244,855,752 Dec 14 16:25 Session.db-wal-rw-r--r-- 1 root root 78,403,632 Dec 16 08:33 Session.db-walrw-r--r-- 1 root root 4,128,272 Dec 17 02:17 Session.db-walGoing back and doing some comparisons and noticed this -
(commas added for 'dramatic effect'
-rw-r--r-- 1 root root 25,082,592 Dec 18 10:39 Session.db-walSession.db-wal is not the right file to be looking at... Should be looking at session.db
-
@Dashrender said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
After speaking with support this morning. Plan is to stop the SC service, rename the DB and start the service to allow the system to rebuild it.
what data will you lose?
The only thing we seem to have lost is Support Sessions, as they are temporary anyway.
-
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@Dashrender said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
After speaking with support this morning. Plan is to stop the SC service, rename the DB and start the service to allow the system to rebuild it.
what data will you lose?
The only thing we seem to have lost is Support Sessions, as they are temporary anyway.
Taht would be expected.
-
So how are we looking after a few hours?
-
@scottalanmiller said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
So how are we looking after a few hours?
We are at 320 of the 540 devices previously logg d.
I expect this as some devices are off,...
-
Any CPU spikes?
-
DB modification was done at 8:00pm(ish) last night, SAR going back to Midnight
]$ sar Linux 3.10.0-327.10.1.el7.x86_64 (server) 12/20/2016 _x86_64_ (1 CPU) 12:00:01 AM CPU %user %nice %system %iowait %steal %idle 12:10:01 AM all 0.42 0.00 0.07 0.02 0.02 99.48 12:20:01 AM all 1.46 0.00 0.16 0.07 0.02 98.29 12:30:01 AM all 0.34 0.00 0.05 0.01 0.01 99.59 12:40:01 AM all 1.02 0.00 0.09 0.08 0.03 98.78 12:50:01 AM all 0.34 0.00 0.05 0.02 0.01 99.58 01:00:01 AM all 1.01 0.00 0.10 0.07 0.01 98.81 01:10:01 AM all 0.64 0.00 0.11 0.02 0.04 99.19 01:20:01 AM all 1.12 0.00 0.09 1.03 0.06 97.71 01:30:01 AM all 0.39 0.00 0.05 0.02 0.03 99.51 01:40:01 AM all 1.06 0.00 0.09 0.07 0.02 98.75 01:50:01 AM all 0.35 0.00 0.05 0.01 0.01 99.57 02:00:01 AM all 0.96 0.00 0.09 0.08 0.02 98.85 02:10:01 AM all 0.57 0.00 0.11 0.02 0.02 99.29 02:20:01 AM all 1.08 0.00 0.10 0.18 0.01 98.62 02:30:01 AM all 0.34 0.00 0.05 0.02 0.02 99.58 02:40:01 AM all 1.02 0.00 0.09 0.07 0.02 98.80 02:50:01 AM all 0.32 0.00 0.05 0.11 0.01 99.51 03:00:01 AM all 1.00 0.00 0.10 0.06 0.02 98.82 03:10:01 AM all 0.39 0.00 0.06 0.02 0.02 99.52 03:20:01 AM all 1.28 0.02 0.15 0.14 0.03 98.38 03:30:01 AM all 0.34 0.00 0.05 0.02 0.02 99.56 03:40:01 AM all 1.05 0.00 0.10 0.08 0.03 98.75 03:50:01 AM all 0.33 0.00 0.06 0.02 0.02 99.58 04:00:01 AM all 1.04 0.00 0.10 0.07 0.03 98.76 04:10:01 AM all 1.07 0.00 0.14 0.04 0.07 98.67 04:20:02 AM all 1.17 0.00 0.13 0.08 0.06 98.57 04:30:01 AM all 0.44 0.31 0.12 0.04 0.04 99.06 04:40:01 AM all 1.07 0.00 0.11 0.08 0.04 98.70 04:50:01 AM all 0.47 0.00 0.07 0.02 0.02 99.42 05:00:01 AM all 1.04 0.00 0.11 0.17 0.05 98.64 05:10:01 AM all 0.76 0.00 0.12 0.04 0.05 99.02 05:20:01 AM all 1.02 0.00 0.09 0.07 0.06 98.76 05:30:01 AM all 0.55 0.00 0.07 0.03 0.04 99.31 05:30:01 AM CPU %user %nice %system %iowait %steal %idle 05:40:01 AM all 1.10 0.00 0.12 0.07 0.03 98.68 05:50:01 AM all 0.55 0.00 0.07 0.03 0.04 99.31 06:00:01 AM all 1.02 0.00 0.09 0.07 0.04 98.78 06:10:01 AM all 1.16 0.00 0.18 0.05 0.07 98.55 06:20:01 AM all 1.43 0.00 0.16 0.09 0.11 98.22 06:30:01 AM all 0.48 0.00 0.06 0.02 0.05 99.39 06:40:01 AM all 1.06 0.00 0.09 0.28 0.07 98.51 06:50:01 AM all 1.13 0.00 0.09 0.03 0.08 98.67 07:00:01 AM all 8.95 0.00 0.17 0.34 0.17 90.37 07:10:01 AM all 3.81 0.00 0.15 0.04 0.18 95.82 07:20:01 AM all 9.25 0.00 0.15 0.17 0.22 90.21 07:30:01 AM all 3.97 0.00 0.10 0.04 0.16 95.72 07:40:01 AM all 10.44 0.00 0.17 0.10 0.29 89.00 07:50:01 AM all 4.62 0.00 0.10 0.05 0.18 95.05 08:00:01 AM all 10.71 0.00 0.15 0.09 0.14 88.91 08:10:01 AM all 5.20 0.00 0.12 0.07 0.22 94.39 Average: all 1.84 0.01 0.10 0.09 0.06 97.90]$ ls -l total 30856 -rw-r--r-- 1 root root 454 Aug 25 19:15 ExtensionConfiguration.xml drwxr-xr-x 2 root root 4096 Nov 8 09:44 Helper -rw-r--r-- 1 root root 654 Aug 25 19:15 License.xml -rw-r--r-- 1 root root 23699 Dec 16 11:26 Role.xml -rw-r--r-- 1 root root 2383872 Dec 20 08:09 Session.db -rw-r--r-- 1 root root 24956928 Dec 19 20:11 Session.db.bak -rw-r--r-- 1 root root 32768 Dec 20 08:16 Session.db-shm -rw-r--r-- 1 root root 4136512 Dec 20 08:16 Session.db-wal -rw-r--r-- 1 root root 4030 Oct 25 08:55 SessionEventTrigger.xml -rw-r--r-- 1 root root 5178 Dec 2 08:13 SessionGroup.xml drwxr-xr-x 14 root root 4096 Dec 12 12:55 Toolbox -rw-r--r-- 1 root root 0 Dec 14 17:24 User.cd60043d-556e-4b23-9ef1-9959e6cea952.xml -rw-r--r-- 1 root root 21908 Dec 19 23:07 User.xmlCurrently 373 devices of the orginial 540
-
I expect that there are so 'few' devices is that they get powered off at the end of the day, and have not been powered on as of yet to call home.
-
Now showing 405 devices connected.
-
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
Now showing 405 devices connected.
How about the CPU?
-
@scottalanmiller said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
Now showing 405 devices connected.
How about the CPU?
-
09:00:01 AM CPU %user %nice %system %iowait %steal %idle 09:10:01 AM all 8.39 0.00 0.28 0.07 0.79 90.48 09:20:01 AM all 14.24 0.00 0.24 0.14 0.79 84.58 09:30:01 AM all 7.18 0.00 0.14 0.08 0.44 92.15 09:40:01 AM all 12.75 0.00 0.23 0.13 0.53 86.35 09:50:01 AM all 15.59 0.00 0.24 0.10 0.36 83.71 10:00:01 AM all 27.39 0.00 0.28 0.10 0.54 71.69 10:10:01 AM all 18.61 0.00 0.24 0.21 0.61 80.33 10:20:01 AM all 24.03 0.00 0.30 0.15 0.71 74.80 10:30:01 AM all 8.70 0.00 0.14 0.09 0.47 90.60 10:40:01 AM all 20.68 0.00 0.27 0.15 0.95 77.95 10:50:01 AM all 10.40 0.00 0.27 0.08 0.80 88.45 Average: all 4.82 0.01 0.13 0.09 0.17 94.78 -
That's higher than I would have expected, but SAR is what we need, not htop.
-
There we go, and still too high.
-
@scottalanmiller said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
There we go, and still too high.
In some regard yes I would agree - However as part of the cycling, I have updated about 15 clients that were on the previous build to the latest. So that could be part of the increased CPU as it was pushing out the update.
-
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@scottalanmiller said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
There we go, and still too high.
In some regard yes I would agree - However as part of the cycling, I have updated about 15 clients that were on the previous build to the latest. So that could be part of the increased CPU as it was pushing out the update.
CPU should not do much while pushing out an update. That's a pretty light CPU activity.
-
@scottalanmiller said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@gjacobse said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
@scottalanmiller said in System Lockups with ScreenConnect 6.0.11622.6115 on CentOS 7:
There we go, and still too high.
In some regard yes I would agree - However as part of the cycling, I have updated about 15 clients that were on the previous build to the latest. So that could be part of the increased CPU as it was pushing out the update.
CPU should not do much while pushing out an update. That's a pretty light CPU activity.
Just received and replied to a followup support email. .cc appropriate persons.
-
File size this morning:
]$ ls -l total 35772 -rw-r--r-- 1 root root 454 Aug 25 19:15 ExtensionConfiguration.xml drwxr-xr-x 2 root root 4096 Nov 8 09:44 Helper -rw-r--r-- 1 root root 654 Aug 25 19:15 License.xml -rw-r--r-- 1 root root 23699 Dec 21 19:15 Role.xml -rw-r--r-- 1 root root 4558848 Dec 22 08:06 Session.db -rw-r--r-- 1 root root 24956928 Dec 19 20:11 Session.db.bak -rw-r--r-- 1 root root 32768 Dec 22 08:18 Session.db-shm -rw-r--r-- 1 root root 6991672 Dec 22 08:18 Session.db-wal -rw-r--r-- 1 root root 4030 Oct 25 08:55 SessionEventTrigger.xml -rw-r--r-- 1 root root 5178 Dec 2 08:13 SessionGroup.xml drwxr-xr-x 14 root root 4096 Dec 12 12:55 Toolbox -rw-r--r-- 1 root root 0 Dec 14 17:24 User.cd60043d-556e-4b23-9ef1-9959e6cea952.xml -rw-r--r-- 1 root root 21905 Dec 21 19:51 User.xmlCPU history:
]$ sar Linux 3.10.0-327.10.1.el7.x86_64 (-) 12/22/2016 _x86_64_ (1 CPU) 12:00:01 AM CPU %user %nice %system %iowait %steal %idle 12:10:01 AM all 1.90 0.00 0.18 0.08 0.15 97.70 12:20:01 AM all 0.74 0.00 0.09 0.03 0.07 99.07 12:30:01 AM all 1.41 0.00 0.10 0.09 0.08 98.33 12:40:01 AM all 0.75 0.00 0.08 0.03 0.09 99.05 12:50:01 AM all 1.42 0.00 0.12 0.08 0.10 98.29 01:00:02 AM all 0.76 0.00 0.09 0.04 0.08 99.04 01:10:01 AM all 1.68 0.00 0.14 0.10 0.20 97.89 01:20:01 AM all 1.05 0.00 0.13 0.04 0.11 98.66 01:30:01 AM all 1.48 0.00 0.12 0.07 0.13 98.20 01:40:01 AM all 0.80 0.00 0.09 0.04 0.10 98.98 01:50:01 AM all 1.45 0.00 0.11 0.07 0.13 98.24 02:00:01 AM all 0.85 0.00 0.07 0.04 0.08 98.96 02:10:01 AM all 1.53 0.00 0.13 0.09 0.18 98.07 02:20:01 AM all 1.04 0.00 0.15 0.04 0.11 98.66 02:30:01 AM all 1.55 0.00 0.13 0.08 0.11 98.13 02:40:02 AM all 0.76 0.00 0.08 0.03 0.10 99.03 02:50:01 AM all 1.47 0.00 0.11 0.08 0.15 98.18 03:00:01 AM all 0.75 0.00 0.08 0.03 0.10 99.04 03:10:01 AM all 1.68 0.00 0.16 0.08 0.20 97.87 03:20:01 AM all 0.69 0.02 0.07 0.03 0.10 99.09 03:30:01 AM all 1.47 0.00 0.10 0.08 0.16 98.20 03:40:01 AM all 0.76 0.00 0.09 0.04 0.12 99.00 03:50:01 AM all 1.61 0.00 0.12 0.10 0.13 98.04 04:00:01 AM all 0.75 0.00 0.07 0.04 0.11 99.03 04:10:01 AM all 2.05 0.00 0.17 0.12 0.18 97.47 04:20:01 AM all 0.82 0.00 0.10 0.11 0.13 98.84 04:30:01 AM all 1.56 0.00 0.12 0.17 0.20 97.95 04:40:01 AM all 0.75 0.00 0.07 0.04 0.15 98.99 04:50:01 AM all 1.43 0.00 0.10 0.08 0.22 98.17 05:00:01 AM all 0.78 0.00 0.08 0.05 0.20 98.90 05:10:01 AM all 1.98 0.00 0.24 0.08 0.25 97.44 05:20:01 AM all 0.85 0.00 0.09 0.04 0.14 98.89 05:30:01 AM all 1.46 0.00 0.11 0.08 0.19 98.17 05:40:01 AM all 0.80 0.00 0.10 0.04 0.15 98.91 05:50:01 AM all 1.72 0.00 0.14 0.12 0.28 97.74 06:00:01 AM all 0.87 0.00 0.09 0.04 0.20 98.80 06:10:01 AM all 1.59 0.00 0.14 0.13 0.26 97.89 06:20:01 AM all 1.03 0.00 0.14 0.05 0.17 98.61 06:30:01 AM all 1.70 0.00 0.13 0.25 0.23 97.69 06:40:01 AM all 0.94 0.00 0.10 0.11 0.17 98.69 06:50:01 AM all 1.58 0.00 0.12 0.14 0.29 97.88 07:00:01 AM all 0.88 0.00 0.10 0.04 0.22 98.76 07:10:01 AM all 1.81 0.00 0.13 0.15 0.38 97.53 07:20:01 AM all 1.44 0.00 0.15 0.11 0.29 98.01 07:30:01 AM all 1.67 0.00 0.11 0.35 0.36 97.52 07:40:01 AM all 0.99 0.00 0.09 0.15 0.22 98.56 07:50:01 AM all 1.76 0.00 0.12 0.34 0.35 97.43 08:00:01 AM all 1.16 0.00 0.10 0.06 0.15 98.53 08:10:01 AM all 1.99 0.00 0.13 0.10 0.19 97.60 Average: all 1.26 0.00 0.11 0.09 0.17 98.36 -
Any lockups since the fix?
