Installing Gluster on CentOS 7
-
@scottalanmiller said:
@johnhooks said:
I'm firing up a couple VMs on my KVM box to test it.
Does Ceph have any advantages? I don't think I can count object storage as an advantage based on what we would be using it for.
Not a lot.
http://www.networkcomputing.com/storage/gluster-vs-ceph-open-source-storage-goes-head-head/8824853
Now that CEPH and Gluster are both inside the RH fold, if you don't want the object flexibility of CEPH, Gluster might be for you.
Ya we would be using it pretty much as a giant NAS. That's what we are experimenting with is older 24 drive servers that were NAS boxes.
-
@scottalanmiller said:
@johnhooks said:
I'm firing up a couple VMs on my KVM box to test it.
Does Ceph have any advantages? I don't think I can count object storage as an advantage based on what we would be using it for.
Not a lot.
http://www.networkcomputing.com/storage/gluster-vs-ceph-open-source-storage-goes-head-head/8824853
Now that CEPH and Gluster are both inside the RH fold, if you don't want the object flexibility of CEPH, Gluster might be for you.
Ha I just read that article like 10 minutes ago.
-
So the next question would be... which IP address do you use for connecting to the Gluster system? the IP address of Brick 1 or Brick 2... or Brick N... ?
Or do you set up some kind of master IP address with Pacemaker / Heartbeat, et al?
-
@dafyre said:
So the next question would be... which IP address do you use for connecting to the Gluster system? the IP address of Brick 1 or Brick 2... or Brick N... ?
Great question. The Gluster client actually handles this. Mount from Server1 and that server fails, the client automatically attaches to Server2. It's not 100% transparent, there is some noticeable delay during the failover but it takes care of itself. It's self healing.
At mount time, you can't do that, if Server1 is down and that's what is in your mount command it can't find the second server. So either you accept that limitation or you put backup servers into the mount command itself and then it handles it at boot time as well.
-
Basically, when mounting, the client appears to query the first node, ask it where the other nodes are, and then is ready to reach out to them as needed. The systems remains able to read and write without any intervention even if an individual node fails.
-
@dafyre said:
So the next question would be... which IP address do you use for connecting to the Gluster system?
Any or all.
-
You forgot
gluster start volume gv0before you mount the volume to
/data -
@scottalanmiller
No package glusterfs-server available ???I tried other articles as well
I can install = centos-release-gluster
but not glusterfs-serve = not available
Oh nvm they changed the url of their repo
Connecting to download.gluster.org (download.gluster.org)|23.253.208.221|:443... connected.
HTTP request sent, awaiting response... 404 Not FoundThis worked for me:
yum search centos-release-gluster #check LTS version number (centos-release-gluster310)
yum -y install centos-release-gluster310 -y
sed -i -e "s/enabled=1/enabled=0/g" /etc/yum.repos.d/CentOS-Gluster-3.10.repo
yum --enablerepo=centos-gluster310,epel -y install glusterfs-server
systemctl start glusterd
systemctl enable glusterd -
@emad-r said in Installing Gluster on CentOS 7:
@scottalanmiller
No package glusterfs-server available ???I tried other articles as well
I can install = centos-release-gluster
but not glusterfs-serve = not available
Oh nvm they changed the url of their repo
Connecting to download.gluster.org (download.gluster.org)|23.253.208.221|:443... connected.
HTTP request sent, awaiting response... 404 Not FoundThis worked for me:
yum search centos-release-gluster #check LTS version number (centos-release-gluster310)
yum -y install centos-release-gluster310 -y
sed -i -e "s/enabled=1/enabled=0/g" /etc/yum.repos.d/CentOS-Gluster-3.10.repo
yum --enablerepo=centos-gluster310,epel -y install glusterfs-server
systemctl start glusterd
systemctl enable glusterdIt's in the storage SIG too. So if you use a mirror local to you, you should be able to find it under storage.
-
I was thinking about doing Gluster Storage for my three KVM Hosts and keep my KVM VMs there. So if I made a virtual machine for the Gluster that used all the storage on each machine and then mounted the Gluster store in each KVM host for storage, would there be any disadvantage to that?
-
@penguinwrangler said in Installing Gluster on CentOS 7:
I was thinking about doing Gluster Storage for my three KVM Hosts and keep my KVM VMs there. So if I made a virtual machine for the Gluster that used all the storage on each machine and then mounted the Gluster store in each KVM host for storage, would there be any disadvantage to that?
Yes, good plan.
That's essentially how many commercial offerings operate today, they just hide the complexity from you.
-
@penguinwrangler said in Installing Gluster on CentOS 7:
I was thinking about doing Gluster Storage for my three KVM Hosts and keep my KVM VMs there. So if I made a virtual machine for the Gluster that used all the storage on each machine and then mounted the Gluster store in each KVM host for storage, would there be any disadvantage to that?
That's Red Hat's HCI model.
-
@scottalanmiller @travisdh1 Another question. I have two SSDs for the main OS (RAID 1), CentOS, then I have the 8 TB enterprise drive for the gluster store. What are your thoughts of needing raid on the 8 TB drive that would be in each machine? I was going to have the gluster store replicate itself to each machine so we only have 8 TB of storage but in theory, we could lose two of the machines and be okay, correct? In a perfect world, I would raid the 8 TB drives with a raid 1 for redundancy, however, this is for my friend who is at a school district that literally doesn't have two pennies to rub together, so the cost of the drives is an issue. He is just now starting to virtualize machines after I have been badgering him forever about it. He picked up some refurbished supermicro servers that we will be using.
-
@penguinwrangler said in Installing Gluster on CentOS 7:
@scottalanmiller @travisdh1 Another question. I have two SSDs for the main OS (RAID 1), CentOS, then I have the 8 TB enterprise drive for the gluster store. What are your thoughts of needing raid on the 8 TB drive that would be in each machine? I was going to have the gluster store replicate itself to each machine so we only have 8 TB of storage but in theory, we could lose two of the machines and be okay, correct? In a perfect world, I would raid the 8 TB drives with a raid 1 for redundancy, however, this is for my friend who is at a school district that literally doesn't have two pennies to rub together, so the cost of the drives is an issue. He is just now starting to virtualize machines after I have been badgering him forever about it. He picked up some refurbished supermicro servers that we will be using.
What you have with the gluster configuration is already a network based triple mirror. Having a local RAID and a gluster setup becomes a waste of resources quick.
-
@travisdh1 That is where my thinking was going, just wanted to make sure I was
on the correct pagegoing down the right path. -
@penguinwrangler said in Installing Gluster on CentOS 7:
I was thinking about doing Gluster Storage for my three KVM Hosts and keep my KVM VMs there. So if I made a virtual machine for the Gluster that used all the storage on each machine and then mounted the Gluster store in each KVM host for storage, would there be any disadvantage to that?
Why I cant visualize this....
So 3 KVM hosts. Node 1/2/3
And you will make VM that will use the storage, on Node 1/2/3. But how will you overcome the cannot use root partitions with Gluster?
Then you will mount the storage again from Node/1/2/3. But what if the VM went down ? is it an SPOF ?
-
@emad-r said in Installing Gluster on CentOS 7:
@penguinwrangler said in Installing Gluster on CentOS 7:
I was thinking about doing Gluster Storage for my three KVM Hosts and keep my KVM VMs there. So if I made a virtual machine for the Gluster that used all the storage on each machine and then mounted the Gluster store in each KVM host for storage, would there be any disadvantage to that?
Why I cant visualize this....
So 3 KVM hosts. Node 1/2/3
And you will make VM that will use the storage, on Node 1/2/3. But how will you overcome the cannot use root partitions with Gluster?
He has a completely separate drive for the Gluster storage. Doesn't even need to deal with partitioning a single drive.
Then you will mount the storage again from Node/1/2/3. But what if the VM went down ? is it an SPOF ?
Well, the likely hood that one of the VMs would go down is less likely than having a hardware issue with one of the nodes.
When a VM or hardware issue comes up, then that one node/VM drops out of the Gluster group. Now you have 2 nodes active instead of 3. When the one that went down is restored, the data is copied back from the two active nodes.
For a bit of a visual example: https://docs.google.com/document/d/17uSypf3QfAH-E9xHY_6brbOOfRYO5QUzrtmm8Y1vhIA/edit?usp=sharing
-
@scottalanmiller Thanks for this post and answering all my questions. @travisdh1 Thanks for answering all my questions as well. Good Thread!
