Installing GrayLog2 on CentOS 7


  • Service Provider

    This is a work in progress.

    GrayLog is the open source competitor to the ELK stack. Like ELK, GrayLog relies on the ElasticSearch database (and additionally on MongoDB.) GrayLog offers some great features missing from ELK, such as user management, but brings us some additional complexities as well and is a bit more difficult to get working on first install. The GrayLog documentation is rather lacking leaving us mostly on our own if we are not prepared to use their premade VM images.

    Build on CentOS 7 on our own gives us more flexibility. We will start making a VM with two CPUs, eight gigs of RAM and, in this example, a terabyte of data storage space. For a demo install 50-100GB is likely more than enough.

    We clone our base VM but add more vCPU and memory.

    An additional storage device will be highly desired.

    #!/bin/bash
    
    cat > /etc/yum.repos.d/elasticsearch.repo <<EOF
    [elasticsearch-1.7]
    name=Elasticsearch repository for 1.7.x packages
    baseurl=http://packages.elastic.co/elasticsearch/1.7/centos
    gpgcheck=1
    gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
    enabled=1
    EOF
    
    cat > /etc/yum.repos.d/mongodb-org-3.2.repo <<EOF
    [mongodb-org-3.2]
    name=MongoDB Repository
    baseurl=https://repo.mongodb.org/yum/redhat/7Server/mongodb-org/3.2/x86_64/
    gpgcheck=0
    enabled=1
    EOF
    
    cd /tmp
    rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-1.3-repository-el7_latest.rpm
    yum -y install wget firewalld epel-release
    yum -y install nginx httpd-tools unzip glances htop java elasticsearch graylog-server graylog-web mongodb-org policycoreutils-python pwgen perl-Digest-SHA net-tools
    semanage port -a -t mongod_port_t -p tcp 27017
    systemctl start firewalld
    systemctl enable firewalld
    
    mv /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml.old
    echo 'network.host: localhost' > /etc/elasticsearch/elasticsearch.yml
    systemctl start elasticsearch
    systemctl enable elasticsearch
    systemctl start mongod
    systemctl enable mongod
    

    You'll need the firewall open for port 9000 by default to see the web interface.

    There are several configuration files that need changes made to get the system up and running and more to get logging happening.

    GrayLog Screenshot


  • Service Provider

    The GrayLog docs have really bad and mostly useless information around getting logs into GrayLog, so working on that now. There is a lot of information missing around this and what is there doesn't match a working install and there is a gap between the two anyway. But their recommended process for sending logs doesn't match their install procedure.



  • The world at large just has to love Scott.. creating all this free documentation for them...

    Any of those companies ever offer to pay you for this?


  • Service Provider

    @Dashrender said:

    The world at large just has to love Scott.. creating all this free documentation for them...

    Any of those companies ever offer to pay you for this?

    Many of these are open source and have no money for that stuff.


  • Service Provider

    Would be good if you finished the documents for one log system before making new ones for another.



  • So I just set up a new Graylog server and it took about 2 minutes (not including download time). They have pre-built VM appliances, distro packages, ansible playbooks, chef recipes, and puppet whatevers ( I forget what they call theirs). I downloaded the appliance and converted the .ova to a .qcow2. Works great. I really enjoy Graylog more than ELK. It has real authentication, much easier to set up, and has notifications built in (unlike ELK which you need to set up Watcher in addition to everything else).


  • Service Provider

    Documenting the OP here as I'm about to make a lot of changes 🙂


    This is a work in progress.

    GrayLog is the open source competitor to the ELK stack. Like ELK, GrayLog relies on the ElasticSearch database (and additionally on MongoDB.) GrayLog offers some great features missing from ELK, such as user management, but brings us some additional complexities as well and is a bit more difficult to get working on first install. The GrayLog documentation is rather lacking leaving us mostly on our own if we are not prepared to use their premade VM images.

    Build on CentOS 7 on our own gives us more flexibility. We will start making a VM with two CPUs, eight gigs of RAM and, in this example, a terabyte of data storage space. For a demo install 50-100GB is likely more than enough.

    We clone our base VM but add more vCPU and memory.

    An additional storage device will be highly desired.

    #!/bin/bash
    
    cat > /etc/yum.repos.d/elasticsearch.repo <<EOF
    [elasticsearch-1.7]
    name=Elasticsearch repository for 1.7.x packages
    baseurl=http://packages.elastic.co/elasticsearch/1.7/centos
    gpgcheck=1
    gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
    enabled=1
    EOF
    
    cat > /etc/yum.repos.d/mongodb-org-3.2.repo <<EOF
    [mongodb-org-3.2]
    name=MongoDB Repository
    baseurl=https://repo.mongodb.org/yum/redhat/7Server/mongodb-org/3.2/x86_64/
    gpgcheck=0
    enabled=1
    EOF
    
    cd /tmp
    rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-1.3-repository-el7_latest.rpm
    yum -y install wget firewalld epel-release
    yum -y install nginx httpd-tools unzip glances htop java elasticsearch graylog-server graylog-web mongodb-org policycoreutils-python pwgen perl-Digest-SHA net-tools
    semanage port -a -t mongod_port_t -p tcp 27017
    systemctl start firewalld
    systemctl enable firewalld
    
    mv /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml.old
    echo 'network.host: localhost' > /etc/elasticsearch/elasticsearch.yml
    systemctl start elasticsearch
    systemctl enable elasticsearch
    systemctl start mongod
    systemctl enable mongod
    

    You'll need the firewall open for port 9000 by default to see the web interface.

    There are several configuration files that need changes made to get the system up and running and more to get logging happening.

    GrayLog Screenshot


  • Service Provider

    Graylog has updated and no longer relies on the old version of ElasticSearch. It will use ElasticSearch 2 now. So time to revisit.



Looks like your connection to MangoLassi was lost, please wait while we try to reconnect.