ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Solved How can I write two separate outputs from one command?

    IT Discussion
    3
    30
    572
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • IRJI
      IRJ
      last edited by

      I am working on creating some custom logging rules for clamav . The default output from a scan is extremely long as it reports everyfile. The default logging doesnt even include timestamp 🤦 so I added that to each entry of the log file. What I really want to capture is FOUND for when an infection is found. and I want to capture Mon Dec 9 19:29:53 UTC 2019Scanned files: 45 which tells me scan was completed and how many files were scanned

      Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/dshield.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-games.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-worm.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-tftp.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-deleted.rules: Html.Trojan.Blackhole-65 FOUND
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-icmp_info.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-web_client.rules: Html.Exploit.CVE_2018_8373-6654754-1 FOUND
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-policy.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-shellcode.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-info.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-smtp.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-pop3.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-current_events.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-attack_response.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-mobile_malware.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/tor.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-p2p.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-web_server.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-telnet.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-dns.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-scan.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-dos.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-malware.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-ftp.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/botcc.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-rpc.rules: OK
Mon Dec  9 19:29:53 UTC 2019/tmp/clamav_test/emerging-snmp.rules: OK
Mon Dec  9 19:29:53 UTC 2019
Mon Dec  9 19:29:53 UTC 2019----------- SCAN SUMMARY -----------
Mon Dec  9 19:29:53 UTC 2019Known viruses: 6594198
Mon Dec  9 19:29:53 UTC 2019Engine version: 0.101.4
Mon Dec  9 19:29:53 UTC 2019Scanned directories: 1
Mon Dec  9 19:29:53 UTC 2019Scanned files: 45
Mon Dec  9 19:29:53 UTC 2019Infected files: 2
Mon Dec  9 19:29:53 UTC 2019Data scanned: 38.73 MB
Mon Dec  9 19:29:53 UTC 2019Data read: 15.07 MB (ratio 2.57:1)
Mon Dec  9 19:29:53 UTC 2019Time: 50.946 sec (0 m 50 s)

      This will capture any infections and write them here
      clamscan -r --exclude=/sys | grep "FOUND" | sed "s/^/$(date)/ " >> /var/log/clamav/alerts/infections.log

      This will capture the Scanned Files at completion of the output.
      clamscan -r --exclude=/sys | grep "Scanned" | sed "s/^/$(date)/ " >> /var/log/clamav/alerts/scans.log

      How the hell can I do that with one command?

      1 Reply Last reply Reply Quote 1
      • stacksofplatesS
        stacksofplates @IRJ
        last edited by

        @IRJ said in How can I write two separate outputs from one command?:

        @stacksofplates said in How can I write two separate outputs from one command?:

        Here's the output from systemd if you create a service:

        Dec 09 15:16:47 localhost.localdomain systemd[1]: Started ClamAV Scanner.
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: **************************************************
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: ***  The virus database is older than 7 days!  ***
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: ***   Please update it as soon as possible.    ***
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: **************************************************
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: ----------- SCAN SUMMARY -----------
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Known viruses: 6561649
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Engine version: 0.101.5
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Scanned directories: 11
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Scanned files: 41
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Infected files: 0
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Data scanned: 32.97 MB
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Data read: 200.09 MB (ratio 0.16:1)
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Time: 30.328 sec (0 m 30 s)
Dec 09 15:17:17 localhost.localdomain systemd[1]: scan.service: Succeeded.

        Can you show me your systemd service file?

        [Unit]
Description=ClamAV Scanner

[Service]
Type=simple
ExecStart=/usr/local/bin/scan.sh

[Install]
WantedBy=default.target

        #!/bin/bash

clamscan -i -r /home/jhooks/Downloads
        1 Reply Last reply Reply Quote 1
        • stacksofplatesS
          stacksofplates
          last edited by

          The only way I know how is to use a script. I wrote a script at the last place for ClamAV to save the output which was in a systemd timer and then a script that you could run to search for what it thought was infections. Can you just shoot this data up into your Elastic Stack and parse it there?

          IRJI 1 Reply Last reply Reply Quote 0
          • IRJI
            IRJ @stacksofplates
            last edited by

            @stacksofplates said in How can I write two separate outputs from one command?:

            The only way I know how is to use a script. I wrote a script at the last place for ClamAV to save the output which was in a systemd timer and then a script that you could run to search for what it thought was infections. Can you just shoot this data up into your Elastic Stack and parse it there?

            Yes I am already doing that from wazuh, but honestly I dont want all these logs. I just want to ship infections and scan completions really. It's a stupid long log file.

            1 Reply Last reply Reply Quote 0
            • stacksofplatesS
              stacksofplates
              last edited by

              @IRJ said in How can I write two separate outputs from one command?:

              sed "s/^/$(date)/ " >> /var/log/clamav/alerts/infections.log

              Let me go back and see if I can find anything on what I did. I think we just output the scan summary but I can't remember off of the top of my head.

              1 Reply Last reply Reply Quote 0
              • stacksofplatesS
                stacksofplates
                last edited by

                Yeah if you just pass a -i it will only print infected files. That's what we did. Here's what the summary looks like then.

                /home/jhooks/Downloads/test.txt: Eicar-Test-Signature FOUND

----------- SCAN SUMMARY -----------
Known viruses: 6561649
Engine version: 0.101.5
Scanned directories: 11
Scanned files: 42
Infected files: 1
Data scanned: 32.97 MB
Data read: 200.09 MB (ratio 0.16:1)
Time: 29.135 sec (0 m 29 s)

                That's the output from clamscan -i -r and just outputting that to a file without grepping.

                IRJI 1 Reply Last reply Reply Quote 2
                • stacksofplatesS
                  stacksofplates
                  last edited by stacksofplates

                  So you get both the location of the infected file(s) and the summary with the number of scanned directories and files. And also the engine version.

                  1 Reply Last reply Reply Quote 0
                  • scottalanmillerS
                    scottalanmiller
                    last edited by

                    Doesn't tee handle this for you?

                    stacksofplatesS 1 Reply Last reply Reply Quote 0
                    • stacksofplatesS
                      stacksofplates @scottalanmiller
                      last edited by stacksofplates

                      @scottalanmiller said in How can I write two separate outputs from one command?:

                      Doesn't tee handle this for you?

                      No it's two separate log outputs. Taht would work if it was the same output. But I don't think he really needs that anyway.

                      1 Reply Last reply Reply Quote 1
                      • IRJI
                        IRJ @stacksofplates
                        last edited by

                        @stacksofplates said in How can I write two separate outputs from one command?:

                        Yeah if you just pass a -i it will only print infected files. That's what we did. Here's what the summary looks like then.

                        /home/jhooks/Downloads/test.txt: Eicar-Test-Signature FOUND

----------- SCAN SUMMARY -----------
Known viruses: 6561649
Engine version: 0.101.5
Scanned directories: 11
Scanned files: 42
Infected files: 1
Data scanned: 32.97 MB
Data read: 200.09 MB (ratio 0.16:1)
Time: 29.135 sec (0 m 29 s)

                        That's the output from clamscan -i -r and just outputting that to a file without grepping.

                        That will work I will just need to add a timestamp. How ridiculous is it that it has no timestamp? lol

                        stacksofplatesS 1 Reply Last reply Reply Quote 0
                        • stacksofplatesS
                          stacksofplates @IRJ
                          last edited by

                          @IRJ said in How can I write two separate outputs from one command?:

                          @stacksofplates said in How can I write two separate outputs from one command?:

                          Yeah if you just pass a -i it will only print infected files. That's what we did. Here's what the summary looks like then.

                          /home/jhooks/Downloads/test.txt: Eicar-Test-Signature FOUND

----------- SCAN SUMMARY -----------
Known viruses: 6561649
Engine version: 0.101.5
Scanned directories: 11
Scanned files: 42
Infected files: 1
Data scanned: 32.97 MB
Data read: 200.09 MB (ratio 0.16:1)
Time: 29.135 sec (0 m 29 s)

                          That's the output from clamscan -i -r and just outputting that to a file without grepping.

                          That will work I will just need to add a timestamp. How ridiculous is it that it has no timestamp? lol

                          ha yeah. We just stuck it in a file named the date. But yeah that's dumb there isn't any dates.

                          IRJI 1 Reply Last reply Reply Quote 0
                          • IRJI
                            IRJ @stacksofplates
                            last edited by

                            @stacksofplates said in How can I write two separate outputs from one command?:

                            @IRJ said in How can I write two separate outputs from one command?:

                            @stacksofplates said in How can I write two separate outputs from one command?:

                            Yeah if you just pass a -i it will only print infected files. That's what we did. Here's what the summary looks like then.

                            /home/jhooks/Downloads/test.txt: Eicar-Test-Signature FOUND

----------- SCAN SUMMARY -----------
Known viruses: 6561649
Engine version: 0.101.5
Scanned directories: 11
Scanned files: 42
Infected files: 1
Data scanned: 32.97 MB
Data read: 200.09 MB (ratio 0.16:1)
Time: 29.135 sec (0 m 29 s)

                            That's the output from clamscan -i -r and just outputting that to a file without grepping.

                            That will work I will just need to add a timestamp. How ridiculous is it that it has no timestamp? lol

                            ha yeah. We just stuck it in a file named the date. But yeah that's dumb there isn't any dates.

                            So you create a separate log file each time? I would think it would be easier to look at a single log file especially since we only have a few line output in this one.

                            stacksofplatesS 1 Reply Last reply Reply Quote 0
                            • stacksofplatesS
                              stacksofplates @IRJ
                              last edited by

                              @IRJ said in How can I write two separate outputs from one command?:

                              @stacksofplates said in How can I write two separate outputs from one command?:

                              @IRJ said in How can I write two separate outputs from one command?:

                              @stacksofplates said in How can I write two separate outputs from one command?:

                              Yeah if you just pass a -i it will only print infected files. That's what we did. Here's what the summary looks like then.

                              /home/jhooks/Downloads/test.txt: Eicar-Test-Signature FOUND

----------- SCAN SUMMARY -----------
Known viruses: 6561649
Engine version: 0.101.5
Scanned directories: 11
Scanned files: 42
Infected files: 1
Data scanned: 32.97 MB
Data read: 200.09 MB (ratio 0.16:1)
Time: 29.135 sec (0 m 29 s)

                              That's the output from clamscan -i -r and just outputting that to a file without grepping.

                              That will work I will just need to add a timestamp. How ridiculous is it that it has no timestamp? lol

                              ha yeah. We just stuck it in a file named the date. But yeah that's dumb there isn't any dates.

                              So you create a separate log file each time? I would think it would be easier to look at a single log file especially since we only have a few line output in this one.

                              We only did that because we had people who didn't know what they were doing looking at the logs. If it was for me, I wouldn't even have it on the systems, but our ISSM wanted the logs on the machines and to have some goons read the log files.

                              1 Reply Last reply Reply Quote 0
                              • stacksofplatesS
                                stacksofplates
                                last edited by

                                If you set this up as a systemd unit/timer, the date will be automatically appended and set up for you. Then you can use journald to read logs. It should give you a lot of flexibility. Then you can scrape them just like system logs in Elastic Stack.

                                1 Reply Last reply Reply Quote 0
                                • IRJI
                                  IRJ
                                  last edited by

                                  @stacksofplates Here is how it looks when I add the date. I cannot same to get a space in between however

                                  clamscan -i -r --exclude=/sys | sed "s/^/$(date)/ " >> /var/log/clamav/scan_log

                                  Mon Dec  9 20:10:59 UTC 2019/tmp/clamav_test/emerging-deleted.rules: Html.Trojan.Blackhole-65 FOUND
Mon Dec  9 20:10:59 UTC 2019/tmp/clamav_test/emerging-web_client.rules: Html.Exploit.CVE_2018_8373-6654754-1 FOUND
Mon Dec  9 20:10:59 UTC 2019
Mon Dec  9 20:10:59 UTC 2019----------- SCAN SUMMARY -----------
Mon Dec  9 20:10:59 UTC 2019Known viruses: 6594198
Mon Dec  9 20:10:59 UTC 2019Engine version: 0.101.4
Mon Dec  9 20:10:59 UTC 2019Scanned directories: 1
Mon Dec  9 20:10:59 UTC 2019Scanned files: 45
Mon Dec  9 20:10:59 UTC 2019Infected files: 2
Mon Dec  9 20:10:59 UTC 2019Data scanned: 38.73 MB
Mon Dec  9 20:10:59 UTC 2019Data read: 15.07 MB (ratio 2.57:1)
Mon Dec  9 20:10:59 UTC 2019Time: 49.446 sec (0 m 49 s)
                                  1 Reply Last reply Reply Quote 0
                                  • stacksofplatesS
                                    stacksofplates
                                    last edited by

                                    Here's the output from systemd if you create a service:

                                    Dec 09 15:16:47 localhost.localdomain systemd[1]: Started ClamAV Scanner.
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: **************************************************
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: ***  The virus database is older than 7 days!  ***
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: ***   Please update it as soon as possible.    ***
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: **************************************************
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: ----------- SCAN SUMMARY -----------
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Known viruses: 6561649
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Engine version: 0.101.5
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Scanned directories: 11
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Scanned files: 41
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Infected files: 0
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Data scanned: 32.97 MB
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Data read: 200.09 MB (ratio 0.16:1)
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Time: 30.328 sec (0 m 30 s)
Dec 09 15:17:17 localhost.localdomain systemd[1]: scan.service: Succeeded.
                                    IRJI 1 Reply Last reply Reply Quote 0
                                    • IRJI
                                      IRJ @stacksofplates
                                      last edited by

                                      @stacksofplates said in How can I write two separate outputs from one command?:

                                      Here's the output from systemd if you create a service:

                                      Dec 09 15:16:47 localhost.localdomain systemd[1]: Started ClamAV Scanner.
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: **************************************************
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: ***  The virus database is older than 7 days!  ***
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: ***   Please update it as soon as possible.    ***
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: **************************************************
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: ----------- SCAN SUMMARY -----------
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Known viruses: 6561649
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Engine version: 0.101.5
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Scanned directories: 11
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Scanned files: 41
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Infected files: 0
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Data scanned: 32.97 MB
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Data read: 200.09 MB (ratio 0.16:1)
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Time: 30.328 sec (0 m 30 s)
Dec 09 15:17:17 localhost.localdomain systemd[1]: scan.service: Succeeded.

                                      Can you show me your systemd service file?

                                      stacksofplatesS 1 Reply Last reply Reply Quote 0
                                      • stacksofplatesS
                                        stacksofplates
                                        last edited by

                                        If it were me, I'd just set up a service and timer. Then it's super easy to automate and audit. You just make sure the service and timer are enabled and you can check whenever you need that they are. Logs are really easy to grab then too. For this I just ran journalctl -u scan

                                        1 Reply Last reply Reply Quote 0
                                        • stacksofplatesS
                                          stacksofplates @IRJ
                                          last edited by

                                          @IRJ said in How can I write two separate outputs from one command?:

                                          @stacksofplates said in How can I write two separate outputs from one command?:

                                          Here's the output from systemd if you create a service:

                                          Dec 09 15:16:47 localhost.localdomain systemd[1]: Started ClamAV Scanner.
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: **************************************************
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: ***  The virus database is older than 7 days!  ***
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: ***   Please update it as soon as possible.    ***
Dec 09 15:16:47 localhost.localdomain scan.sh[23673]: LibClamAV Warning: **************************************************
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: ----------- SCAN SUMMARY -----------
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Known viruses: 6561649
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Engine version: 0.101.5
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Scanned directories: 11
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Scanned files: 41
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Infected files: 0
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Data scanned: 32.97 MB
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Data read: 200.09 MB (ratio 0.16:1)
Dec 09 15:17:17 localhost.localdomain scan.sh[23673]: Time: 30.328 sec (0 m 30 s)
Dec 09 15:17:17 localhost.localdomain systemd[1]: scan.service: Succeeded.

                                          Can you show me your systemd service file?

                                          [Unit]
Description=ClamAV Scanner

[Service]
Type=simple
ExecStart=/usr/local/bin/scan.sh

[Install]
WantedBy=default.target

                                          #!/bin/bash

clamscan -i -r /home/jhooks/Downloads
                                          1 Reply Last reply Reply Quote 1
                                          • stacksofplatesS
                                            stacksofplates
                                            last edited by stacksofplates

                                            A timer would just be this:

                                            [Unit]
Description=Run Clam Scan

[Timer]
OnCalendar=*-*-* 00:00:00
Unit=scan.service

[Install]
WantedBy=default.target
                                            IRJI 1 Reply Last reply Reply Quote 1
                                            • IRJI
                                              IRJ @stacksofplates
                                              last edited by

                                              @stacksofplates said in How can I write two separate outputs from one command?:

                                              A timer would just be this:

                                              [Unit]
Description=Run Clam Scan

[Timer]
OnCalendar=*-*-* 00:00:00
Unit=scan.service

[Install]
WantedBy=default.target

                                              Do you run systemctl enable clamav.timer and systemclt start clamav.timer instead of doing it with service?

                                              stacksofplatesS 1 Reply Last reply Reply Quote 0
                                              • 1
                                              • 2
                                              • 1 / 2
                                              • First post
                                                Last post