ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    GlusterFS + WebDAV Centos Setup Guide

    IT Discussion
    glusterfs winscp centos replication webdav
    1
    1
    1.5k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Emad RE
      Emad R
      last edited by Emad R

      This will be a quick and dirty setup guide, please if you see something stupid point it out. I started from the latest Centos 7.3 minimal and I applied this on 3 nodes. This helped me understand how NAS works, and it is very rough draft. I felt bad not writing it once I learned this cause the documentation now i have is 1 month old and I it may be lacking, thus I apologize. before hand. Please read this guide with the notion of understanding and not copy/paste implementing. I mean after you understand the goal below and understand it I reckon you can implement it better by enhancing it.

      I recommend before you start to partition Centos in such a way:

      2 GiB boot | 50 GiB root | 2 GiB swap

      Cause GlusterFS needs to be away from root / partition.
      And leave 1 empty unallocated partition/space for Gluster, we will create in latter steps.

      Before we start, the imaginary goal I created is create 3 nodes that replicate the Filesystem, and if 1 node can go down, users can have the option to connect to node 2 or 3 (manually by entering different IP address), and users will be using WinSCP (cause it rocks in stability and security and performance also it tries to make it easy for dumb users). So let us start:

      yum search centos-release-gluster  	# check LTS version  (if you find  latest + LTS install)
yum -y install centos-release-gluster310 -y
sed -i -e "s/enabled=1/enabled=0/g" /etc/yum.repos.d/CentOS-Gluster-3.10.repo
yum --enablerepo=centos-gluster310,epel -y install glusterfs-server
systemctl enable glusterd
systemctl start rpcbind
systemctl enable rpcbind
systemctl restart glusterd

      Part 1 finished, we installed and started required packages and services.

      Prep the servers/nodes by changing their hostnames and configured /etc/hosts to reflect that, for example ensure hostname is something like centos.gluster.1 | centos.gluster.2 | centos.gluster.3 (one on each node)

      nano /etc/hostname

      Then:

      nano /etc/hosts

      And put the hostnames to manually point to IP of their respective machines, in each of the nodes

      	192.168.1.20 centos.gluster.1
	192.168.1.19 centos.gluster.2
	192.168.1.18 centos.gluster.3

      Now on all nodes do this:

      mkdir -p /glusterfs
mkdir -p /replica_files

      Then use cfdisk to create/write the new partition without a type. it has command line interface simply navigate using the arrow keys and create partition.

      Then run the below on all 3 nodes

      partprobe
mkfs.xfs /dev/sda4                                (replace /dev/sda4 with the your partition)
partprobe
mount /dev/sda4 /glusterfs/
nano /etc/fstab -> /dev/sda4       /glusterfs      xfs     defaults 0 0
mkdir -p /glusterfs/replicafs

      Part 2 done we created the folder structure and prepared the filesystem

      Now let us connect the peers:

      gluster peer probe centos.gluster.1	 #(do this from node 1/2/3)
gluster peer probe centos.gluster.2	 #(do this from node 1/2/3)
gluster peer probe centos.gluster.3	 #(do this from node 1/2/3)

      and check that all is good by running:

      gluster peer status

      If you have any issues, trying pinging the hostnames from the nodes and see if they translate to IP properly, if not check your hosts file.

      Let us create the replication between 3 nodes, you need to run this on 1 node only

      gluster volume create vol_replica replica 3 transport tcp \	
centos.gluster.1:/glusterfs/replicafs \
centos.gluster.2:/glusterfs/replicafs \
centos.gluster.3:/glusterfs/replicafs
gluster volume start vol_replica
gluster volume info

      You can then mount this replicafs by running the below on each node respectively :

      mount -t glusterfs centos.gluster.1:/vol_replica  /replica_files (run only on node 1)
mount -t glusterfs centos.gluster.2:/vol_replica  /replica_files (run only on node 2)
mount -t glusterfs centos.gluster.3:/vol_replica  /replica_files (run only on node 3)

      I had trouble using /etc/fstab to mount the above 3 commands, so I just scheduled the above commands to run at startup.

      Now the fun starts with WebDAV you can do this on seperate node or on one of the 3 nodes or on all 3 nodes, it is up to you.

      yum -y install httpd
systemctl start httpd 
systemctl enable httpd

      Create SSL/TLS key + Certificate + Certificate Signing request (3 files server.key|server.cert|server.csr)

      cd /etc/pki/tls/certs
make server.key 
openssl rsa -in server.key -out server.key 
make server.csr 
openssl x509 -in server.csr -out server.crt -req -signkey server.key -days 365


yum -y install mod_ssl
nano /etc/httpd/conf.d/ssl.conf
	# line 59: uncomment -> DocumentRoot "/var/www/html"
	# line 60: uncomment and specify the server name -> ServerName 192.168.1.20:443
	# line 100: change to the one created earlier -> SSLCertificateFile /etc/pki/tls/certs/server.crt
	# line 107: change to the one created earlier -> SSLCertificateKeyFile /etc/pki/tls/certs/server.key
	# line 110: comment out -> # SSLCertificateChainFile

      To Change SSL/TLS 443 default port, edit 3 lines:

      nano /etc/httpd/conf.d/ssl.conf
Listen 7777 https
<VirtualHost _default_:7777>
ServerName 192.168.1.20:7777

      Using WebDAV + GlusterFS:

      make sure you mounted the glusterfs on /replica_files first

      mkdir /replica_files/public
chown apache. /replica_files/public
chmod 770 /replica_files/public

      nano /etc/httpd/conf.d/webdav.conf

      	DavLockDB "/tmp/DavLock"
	Alias /public /replica_files/public
	<Location /public>
	DAV On
	SSLRequireSSL
	Options None
	AuthType Basic
	AuthName "Public Share"
	AuthUserFile /etc/httpd/conf/.htpasswd
	<RequireAny>
    Require method GET POST OPTIONS
    Require valid-user
	</RequireAny>
	</Location>

      Create user

      htpasswd -B -C 10 /etc/httpd/conf/.htpasswd jane	# Create Bcrypt strong pass for jane (Works with WinSCP)

      What if you want to create a personal share, not public one, will you need to understand a public share needs to be accessed with username and pass, but you can share this credentials to multiple users.

      However you can also separate database password files, which adds separation layer, for example even if Jane password is correct she wont access another folder (hr folder for example).

      mkdir -p /replica_files/hr
chown apache. /replica_files/hr
chmod 770 /replica_files/hr

      nano /etc/httpd/conf.d/webdav.conf

      And add:

      Alias /hr /replica_files/hr
<Directory /replica_files/hr>
    DAV             On
    AuthType        Basic 
    AuthName        "HR Private Share"
    AuthUserFile    /etc/httpd/conf/hr.passwd
    Require         valid-user

      The only noteable change is hr.passwd instead of .htpasswd
      to create user there (hr.passwd):

      htpasswd -c -B -C 10 /etc/httpd/conf/hr.passwd marc

      add another user

      htpasswd -B -C 10 /etc/httpd/conf/hr.passwd marie

      dont use the -c twice it will re-create the database file. Other useful commands include:

      htpasswd -v	/etc/httpd/conf/.htpasswd jane			# to verify user exist
htpasswd -D	/etc/httpd/conf/.htpasswd jane			# delete user jane

      htpasswd password encryption info:

      -m default is MD5 htpasswd
-B is Bcrypt which is very secure can be paired with -C valid values 4-31 (higher is more secure but slower)
-s SHA encryption (insecure)

      If SELinux is enabled, change rules like follows.

      chcon -R -t httpd_sys_rw_content_t /webdav_folder_location
semanage fcontext -a -t httpd_sys_rw_content_t /webdav_folder_location

      In WinSCP you can connect using one liner:

      https://marc:[email protected]:7777/hr/

      copy and paste this in Hostname

      In the end you can create structure, if you installed Apace+WebDav on all 3 nodes, that if one node goes down you can instruct users to use node 2 IP address, and it will have exactly the same files. but users need to know the 3 nodes Address before hand, I still didnt know how to group all 3 nodes location behind load balancing proxy or something similar. will learn that in the future.

      Many of the article especially GlusterFs take from guides already found in ML, see scott guide:
      https://mangolassi.it/topic/8619/installing-gluster-on-centos-7/1

      1 Reply Last reply Reply Quote 3
      • 1 / 1
      • First post
        Last post