Installing GrayLog2 on CentOS 7

scottalanmiller

This is a work in progress.

GrayLog is the open source competitor to the ELK stack. Like ELK, GrayLog relies on the ElasticSearch database (and additionally on MongoDB.) GrayLog offers some great features missing from ELK, such as user management, but brings us some additional complexities as well and is a bit more difficult to get working on first install. The GrayLog documentation is rather lacking leaving us mostly on our own if we are not prepared to use their premade VM images.

Build on CentOS 7 on our own gives us more flexibility. We will start making a VM with two CPUs, eight gigs of RAM and, in this example, a terabyte of data storage space. For a demo install 50-100GB is likely more than enough.

We clone our base VM but add more vCPU and memory.

An additional storage device will be highly desired.

#!/bin/bash

cat > /etc/yum.repos.d/elasticsearch.repo <<EOF
[elasticsearch-1.7]
name=Elasticsearch repository for 1.7.x packages
baseurl=http://packages.elastic.co/elasticsearch/1.7/centos
gpgcheck=1
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1
EOF

cat > /etc/yum.repos.d/mongodb-org-3.2.repo <<EOF
[mongodb-org-3.2]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/7Server/mongodb-org/3.2/x86_64/
gpgcheck=0
enabled=1
EOF

cd /tmp
rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-1.3-repository-el7_latest.rpm
yum -y install wget firewalld epel-release
yum -y install nginx httpd-tools unzip glances htop java elasticsearch graylog-server graylog-web mongodb-org policycoreutils-python pwgen perl-Digest-SHA net-tools
semanage port -a -t mongod_port_t -p tcp 27017
systemctl start firewalld
systemctl enable firewalld

mv /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml.old
echo 'network.host: localhost' > /etc/elasticsearch/elasticsearch.yml
systemctl start elasticsearch
systemctl enable elasticsearch
systemctl start mongod
systemctl enable mongod

You'll need the firewall open for port 9000 by default to see the web interface.

There are several configuration files that need changes made to get the system up and running and more to get logging happening.

GrayLog Screenshot

scottalanmiller

The GrayLog docs have really bad and mostly useless information around getting logs into GrayLog, so working on that now. There is a lot of information missing around this and what is there doesn't match a working install and there is a gap between the two anyway. But their recommended process for sending logs doesn't match their install procedure.

Dashrender

The world at large just has to love Scott.. creating all this free documentation for them...

Any of those companies ever offer to pay you for this?

scottalanmiller

@Dashrender said:

The world at large just has to love Scott.. creating all this free documentation for them...

Any of those companies ever offer to pay you for this?

Many of these are open source and have no money for that stuff.

JaredBusch

Would be good if you finished the documents for one log system before making new ones for another.

stacksofplates

So I just set up a new Graylog server and it took about 2 minutes (not including download time). They have pre-built VM appliances, distro packages, ansible playbooks, chef recipes, and puppet whatevers ( I forget what they call theirs). I downloaded the appliance and converted the .ova to a .qcow2. Works great. I really enjoy Graylog more than ELK. It has real authentication, much easier to set up, and has notifications built in (unlike ELK which you need to set up Watcher in addition to everything else).

scottalanmiller

Documenting the OP here as I'm about to make a lot of changes