ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Building ELK on CentOS 7

    Scheduled Pinned Locked Moved IT Discussion
    scalentg labscale hc3centoscentos 7elklogginglog managementhow tolinuxelasticsearchkibanalogstashkibana 4
    43 Posts 8 Posters 16.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • JaredBuschJ
      JaredBusch @scottalanmiller
      last edited by

      @scottalanmiller so what do you setup your disk partitioning like in CentOS 7?

      On a minimal install left to automatic, if you use a larger drive, it will create a separate partition for all the space after 50gb.

      this is highly annoying because I created a 127GB drive (default in Hyper-V) and now 50GB is separate from all the rest.

      scottalanmillerS 1 Reply Last reply Reply Quote 0
      • JaredBuschJ
        JaredBusch
        last edited by

        like this

        [root@elk ~]# df -h
        Filesystem                   Size  Used Avail Use% Mounted on
        /dev/mapper/centos_elk-root   50G  855M   50G   2% /
        devtmpfs                     906M     0  906M   0% /dev
        tmpfs                        916M     0  916M   0% /dev/shm
        tmpfs                        916M  8.3M  907M   1% /run
        tmpfs                        916M     0  916M   0% /sys/fs/cgroup
        /dev/sda2                    494M   98M  396M  20% /boot
        /dev/sda1                    200M  9.5M  191M   5% /boot/efi
        /dev/mapper/centos_elk-home   75G   33M   75G   1% /home
        tmpfs                        184M     0  184M   0% /run/user/0
        [root@elk ~]#
        
        1 Reply Last reply Reply Quote 0
        • JaredBuschJ
          JaredBusch
          last edited by

          You should at least tell the user that you are asking for the kibana password.

          htpasswd -c /etc/nginx/htpasswd.users kibanauser

          0_1456293132171_upload-2a181928-c672-4286-86d6-43b69bd92fc3

          1 Reply Last reply Reply Quote 0
          • JaredBuschJ
            JaredBusch
            last edited by

            I had this error.

            0_1456293268870_upload-fc8bbfa4-c8e3-4c86-8e10-5cd5cf3195c8

            1 Reply Last reply Reply Quote 0
            • JaredBuschJ
              JaredBusch
              last edited by

              Looks like maybe you forgot to start firewalld?

                % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                               Dload  Upload   Total   Spent    Left  Speed
              100   814  100   814    0     0   1370      0 --:--:-- --:--:-- --:--:--  1372
              {
                "acknowledged" : true
              }
              FirewallD is not running
              FirewallD is not running
              [root@elk ~]# yum install firewalld
              Loaded plugins: fastestmirror
              Loading mirror speeds from cached hostfile
               * base: mirror.oss.ou.edu
               * epel: fedora-epel.mirror.lstn.net
               * extras: centos.mirrors.wvstateu.edu
               * updates: centos.mirrors.wvstateu.edu
              Package firewalld-0.3.9-14.el7.noarch already installed and latest version
              Nothing to do
              [root@elk ~]# systemctl start firewalld
              [root@elk ~]# systemctl status firewalld
              ● firewalld.service - firewalld - dynamic firewall daemon
                 Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
                 Active: active (running) since Tue 2016-02-23 23:55:11 CST; 14s ago
               Main PID: 11482 (firewalld)
                 CGroup: /system.slice/firewalld.service
                         └─11482 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid
              
              Feb 23 23:55:09 elk systemd[1]: Starting firewalld - dynamic firewall daemon...
              Feb 23 23:55:11 elk systemd[1]: Started firewalld - dynamic firewall daemon.
              [root@elk ~]#
              
              scottalanmillerS 1 Reply Last reply Reply Quote 0
              • JaredBuschJ
                JaredBusch
                last edited by

                Yeah, you set it to install, but you never start or enable it.

                0_1456293489801_upload-2c71d7be-5435-4f22-ab21-c2912aa344c8

                1 Reply Last reply Reply Quote 0
                • JaredBuschJ
                  JaredBusch
                  last edited by JaredBusch

                  Line 109 needs commented out.

                  0_1456293589646_upload-722d8a55-ede0-467f-815e-97aca00bde17

                  add this right after the yum install to fix the firewall.

                  yum -y install wget firewalld epel-release
                  systemctl enable firewalld
                  systemctl start firewalld
                  yum -y install nginx httpd-tools unzip
                  

                  I would just remove line 109 it serves no purpose.

                  Edit: Some dumbass forgot to snapshot the image so he could repeat the install...

                  scottalanmillerS 1 Reply Last reply Reply Quote 0
                  • JaredBuschJ
                    JaredBusch
                    last edited by JaredBusch

                    Why lock out with .htaccess? There is no hint what is needed to log in here.

                    0_1456293980571_upload-16198c5d-89fa-48ac-a702-3d6b2cc05644

                    I hate this level of authentication.

                    Using kibanauser and the password I chose, results in Kibana setup.
                    0_1456294107205_upload-68bec54b-23aa-4026-95d9-8080cfed408d

                    scottalanmillerS 1 Reply Last reply Reply Quote 0
                    • scottalanmillerS
                      scottalanmiller @JaredBusch
                      last edited by

                      @JaredBusch said:

                      @scottalanmiller so what do you setup your disk partitioning like in CentOS 7?

                      If I'm doing this for product, I do 20GB for the OS and 200GB+ on a second VHD for the data. I put it all under LVM and make a XFS filesystem on the secondary mount and mount it to data and make a symlink for the Elasticsearch database directory into there.

                      JaredBuschJ 1 Reply Last reply Reply Quote 0
                      • scottalanmillerS
                        scottalanmiller @JaredBusch
                        last edited by

                        @JaredBusch said:

                        Why lock out with .htaccess? There is no hint what is needed to log in here.

                        It's how Digital Ocean does it as well. Kibana doesn't have a built in authentication scheme that I know of. HTAccess is very simple for someone to just get started.

                        1 Reply Last reply Reply Quote 0
                        • scottalanmillerS
                          scottalanmiller
                          last edited by

                          And simple to remove when you want to move to something else.

                          1 Reply Last reply Reply Quote 0
                          • scottalanmillerS
                            scottalanmiller @JaredBusch
                            last edited by

                            @JaredBusch said:

                            Line 109 needs commented out.

                            0_1456293589646_upload-722d8a55-ede0-467f-815e-97aca00bde17

                            add this right after the yum install to fix the firewall.

                            yum -y install wget firewalld epel-release
                            systemctl enable firewalld
                            systemctl start firewalld
                            yum -y install nginx httpd-tools unzip
                            

                            I would just remove line 109 it serves no purpose.

                            Edit: Some dumbass forgot to snapshot the image so he could repeat the install...

                            Thanks. That was formatting I had originally put in before scripting it.

                            1 Reply Last reply Reply Quote 0
                            • scottalanmillerS
                              scottalanmiller @JaredBusch
                              last edited by

                              @JaredBusch said:

                              Looks like maybe you forgot to start firewalld?

                              Fixed

                              1 Reply Last reply Reply Quote 0
                              • JaredBuschJ
                                JaredBusch @scottalanmiller
                                last edited by

                                @scottalanmiller said:

                                @JaredBusch said:

                                @scottalanmiller so what do you setup your disk partitioning like in CentOS 7?

                                If I'm doing this for product, I do 20GB for the OS and 200GB+ on a second VHD for the data. I put it all under LVM and make a XFS filesystem on the secondary mount and mount it to data and make a symlink for the Elasticsearch database directory into there.

                                SO this mean you need to make one of your linux admin setrups on drive settings because that is not what CentOS does by dfault.

                                DashrenderD 1 Reply Last reply Reply Quote 0
                                • DashrenderD
                                  Dashrender @JaredBusch
                                  last edited by

                                  @JaredBusch said:

                                  @scottalanmiller said:

                                  @JaredBusch said:

                                  @scottalanmiller so what do you setup your disk partitioning like in CentOS 7?

                                  If I'm doing this for product, I do 20GB for the OS and 200GB+ on a second VHD for the data. I put it all under LVM and make a XFS filesystem on the secondary mount and mount it to data and make a symlink for the Elasticsearch database directory into there.

                                  SO this mean you need to make one of your linux admin setrups on drive settings because that is not what CentOS does by dfault.

                                  Would CentOS do what Scott does if you had two drives you provide CentOS to use? i.e. a 20 GB and a 200+ GB one? Would CentOS install the OS and everything fully on the 20, and then just mount the 200 on some point?

                                  JaredBuschJ 1 Reply Last reply Reply Quote 0
                                  • JaredBuschJ
                                    JaredBusch @Dashrender
                                    last edited by

                                    @Dashrender said:

                                    @JaredBusch said:

                                    @scottalanmiller said:

                                    @JaredBusch said:

                                    @scottalanmiller so what do you setup your disk partitioning like in CentOS 7?

                                    If I'm doing this for product, I do 20GB for the OS and 200GB+ on a second VHD for the data. I put it all under LVM and make a XFS filesystem on the secondary mount and mount it to data and make a symlink for the Elasticsearch database directory into there.

                                    SO this mean you need to make one of your linux admin setrups on drive settings because that is not what CentOS does by dfault.

                                    Would CentOS do what Scott does if you had two drives you provide CentOS to use? i.e. a 20 GB and a 200+ GB one? Would CentOS install the OS and everything fully on the 20, and then just mount the 200 on some point?

                                    Than answer is not by default. It tries to make it's own magic.

                                    You can see here I created a 20gb and a 200GB vhdx and told the install to handle it all for me.

                                    0_1456324763628_upload-a8516602-b5e4-4c0e-9112-caabbb970b80

                                    Guess what, you still end up with a 50GB and a 170GB partitions scheme

                                    [root@elk ~]# df -h
                                    Filesystem                   Size  Used Avail Use% Mounted on
                                    /dev/mapper/centos_elk-root   50G  882M   50G   2% /
                                    devtmpfs                     906M     0  906M   0% /dev
                                    tmpfs                        916M     0  916M   0% /dev/shm
                                    tmpfs                        916M  8.3M  907M   1% /run
                                    tmpfs                        916M     0  916M   0% /sys/fs/cgroup
                                    /dev/sda2                    494M   99M  395M  21% /boot
                                    /dev/sda1                    200M  9.5M  191M   5% /boot/efi
                                    /dev/mapper/centos_elk-home  168G   33M  168G   1% /home
                                    tmpfs                        184M     0  184M   0% /run/user/0
                                    [root@elk ~]#
                                    
                                    1 Reply Last reply Reply Quote 0
                                    • JaredBuschJ
                                      JaredBusch
                                      last edited by

                                      CentOS 7 has a thing for 50GB root mounts.

                                      1 Reply Last reply Reply Quote 0
                                      • scottalanmillerS
                                        scottalanmiller
                                        last edited by

                                        Yeah, the defaults suck a bit.

                                        1 Reply Last reply Reply Quote 0
                                        • JaredBuschJ
                                          JaredBusch
                                          last edited by JaredBusch

                                          @scottalanmiller Why are you using Oracle's Java SDK and not just java from the repo?

                                          I had read in another write up on the install that it works fine even if it is not the "official" method.

                                          scottalanmillerS 1 Reply Last reply Reply Quote 0
                                          • scottalanmillerS
                                            scottalanmiller @JaredBusch
                                            last edited by

                                            @JaredBusch said:

                                            @scottalanmiller Why are you using Oracle's Java SDK and not just java from the repo?

                                            I had read in another write up on the install that it works fine even if it is not the "official" method.

                                            Even if it "works", Elasticsearch tests against and only officially supports the Oracle one. Just because I can get the OpenJDK to work, I'd hate to have it be buggy or problematic for someone down the line because I used one that wasn't tested against.

                                            JaredBuschJ 1 Reply Last reply Reply Quote 1
                                            • 1
                                            • 2
                                            • 3
                                            • 1 / 3
                                            • First post
                                              Last post