Setting Up Logstash for ELK


  • Service Provider

    If you use nearly any logstash-forwarder documentation that you find, you might find that the information around CentOS and RHEL is a bit out of date. This can make things very challenging. The most popular documentation around is from Digital Ocean and their Ubuntu docs have been updated but their CentOS have not. With a little work and research I was able to come up with a script that does everything that you need for CentOS and RHEL.

    #!/bin/bash
    #Set Up ELK
    
    mkdir -p /etc/pki/tls/certs
    echo '-----BEGIN CERTIFICATE-----
    ....contents of your cert go here.....
    -----END CERTIFICATE-----' > /etc/pki/tls/certs/logstash-forwarder.crt
    
    wget https://download.elastic.co/logstash-forwarder/binaries/logstash-forwarder-0.4.0-1.x86_64.rpm
    yum -y install logstash-forwarder-0.4.0-1.x86_64.rpm
    rm logstash-forwarder-0.4.0-1.x86_64.rpm
    
    echo '{
      "network": {
        "servers": [ "1.2.3.4:5000" ],
        "timeout": 15,
        "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt"
      },
      "files": [
        {
          "paths": [
            "/var/log/messages",
            "/var/log/security"
           ],
          "fields": { "type": "syslog" }
        }
       ]
    }' > /etc/logstash-forwarder.conf
    
    chkconfig --add logstash-forwarder
    service logstash-forwarder start
    

    You will need to paste in the contents of your own key, of course and the 1.2.3.4 needs to be changed to your ELK's IP address. But other than that, you can just use this script and you are ready to go. Tested on CentOS 6 and CentOS 7. This will get basic logs flowing into a Digital Ocean style ELK install that is currently up to date (Kibana 4 era.)


  • Service Provider

    If you have older CentOS 5 or RHEL 5 you can still use the script, this has been tested and all that we need to is remove the final line of the script and replace it with...

    /etc/init.d/logstash-forwarder start
    

    And now you have CentOS 5, 6 and 7 all supported.


  • Service Provider

    I believe these two commands are not available by default in CentOS 7.

    chkconfig --add logstash-forwarder
    service logstash-forwarder start
    

    should be

    systemctl enable logstash-forwarder
    systemctl start logstash-forwarder

  • Service Provider

    @JaredBusch said:

    I believe these two commands are not available by default in CentOS 7.

    chkconfig --add logstash-forwarder
    service logstash-forwarder start
    

    should be

    systemctl enable logstash-forwarder
    systemctl start logstash-forwarder
    

    Those are the proper ones, definitely. The old ones still work in 7, probably gone in 8, though. The script worked repeatedly on several 7 machines. We only have two 6s left in the fleet, I think.



  • So I just finished installing again on CentOS 7. I'm having an issue which is the same one I had before.

    0_1453412822216_elk.png

    I have a suspicion that it has to do with filebeat. I used Digital Ocean's doc back in October or so and everything worked fine. Now that they switched to filebeat it's not working.


  • Service Provider

    Great. Going to have to do some research. I am planning on building a new one soon myself.



  • So it's not SELinux or firewalld, both are completely off on both the client and the ELK server.



  • Eh I'm just a giant moron. Port 5044 wasn't open, and stopping firewalld doesn't stop the firewall (idiot) so now it's working.

    They don't do that in the documentation though, so just remember to do it. I remembered about port 80 for kibana, but forgot about logstash.



  • They also forget about SELinux with their CentOS 7 docs. You need sudo setsebool -P httpd_can_network_connect 1 and possibly sudo chcon -R --type=httpd_syscontent_rw_t /opt/kibana

    Up and running now.



Looks like your connection to MangoLassi was lost, please wait while we try to reconnect.