KVM Poor Man Replication HA
-
@travisdh1 said in KVM Poor Man Replication HA:
@matteo-nunziati said in KVM Poor Man Replication HA:
gluster has been choosen upstream and by read hat for their hyperconverged stuff, so it should be ok, even if use it without ovirt seems a bit complex. But it is just a feeling, I never used it, just had a look to the how-tos
Yeah, if you don't break it down it can look like a bear, but it's really not that difficult. I haven't spun up a cluster using it in years, otherwise I'd do a how-to here.
I think that I have one.
-
@scottalanmiller said in KVM Poor Man Replication HA:
@travisdh1 said in KVM Poor Man Replication HA:
@matteo-nunziati said in KVM Poor Man Replication HA:
gluster has been choosen upstream and by read hat for their hyperconverged stuff, so it should be ok, even if use it without ovirt seems a bit complex. But it is just a feeling, I never used it, just had a look to the how-tos
Yeah, if you don't break it down it can look like a bear, but it's really not that difficult. I haven't spun up a cluster using it in years, otherwise I'd do a how-to here.
I think that I have one.
https://mangolassi.it/topic/8619/installing-gluster-on-centos-7
Yes you do, and I read it. And looking forward to playing with it. But the thing is my simple method works if you can afford some time lost between both servers, some small workloads can especially if there are separate file\DB backups.Have fun all at MangoCon
What I really want to say and express, its very fun to have full fledged OS to play with that have an hypervisor role, is it very hard for me to go back to ESXi standalone, having a full OS (Centos + KVM) makes a world of possibilities.
-
@travisdh1 said in KVM Poor Man Replication HA:
@matteo-nunziati said in KVM Poor Man Replication HA:
gluster has been choosen upstream and by read hat for their hyperconverged stuff, so it should be ok, even if use it without ovirt seems a bit complex. But it is just a feeling, I never used it, just had a look to the how-tos
Yeah, if you don't break it down it can look like a bear, but it's really not that difficult. I haven't spun up a cluster using it in years, otherwise I'd do a how-to here.
what made me think about complaxity is that they suggest to build only a single brick on top of only a single thin provisioned LVM volume on top of only a single thin provisioned volume.
I mean: everytime you need a brick you need to create a lot of stuff... maybe a shell script can make it more streamlined but if you use plain LVM as virtual disk for your VM it means a lot of stuff everytime you create a VM. boring
-
I deployed it for an HCI cluster (well tried). The issue was it locked read only on files on re-sync operations and certain things (which blew them up). I'm sure it's some ways, but Gluster wasn't designed for VM storage (Neither was Ceph, but it was designed to perform better at writes than gluster). The original core use cases were scaled out file services to get around limitations of scaling of XFS/EXT4 etc and do a little DRAM caching for read/metadata on nodes. It was a poor man's Isilon more than a low latency transactional filer, or general purpose block platform.
The other thing is to be careful with Gluster and quorum. It's REALLY easy to setup. It's also REALLY easy to setup something like what this guy had that was working till it didn't.
-
@matteo-nunziati RedHat's building a GUI for this stuff was my understanding. The other thing to be aware of is running bare metal requires you deal with things like Driver/Firmware lifecycle for Drives, Controller Backplanes and work with the OEM on issues like SAS buffering incompatibility, liberal interpretations of the T10 SPEC, or good old SATA tunneling protocol locking an entire PHY because of a SMART poll and crashing the controller. There be dragons in running storage devices in pass thru mode (especially modern flash devices in dense configs). If you go down this route I'd want to have RedHat or SuSE backing me from a support stance.
-
@aaronstuder said in KVM Poor Man Replication HA:
Ceph would be a better option
I think Redhat knows why they aren't using it. Its core feature set was high-speed transactional object and not block (the Block have was explicitly not production ready when they acquired it). While it is architecturally better for the block, It's dynamic failure handling is non-existent (Gluster is a bit better here because of the layering it tends to sit on and use of Local RAID which was what RedHat was packaging for HCI). Ceph is the #1 reason I've seen for OpenStack deployments to crash and burn in a disastrous way. With Careful testing, validation, lifecycle control, and prescriptive deployments it's powerful. It's just... Kinda a glass cannon, and you'll see it openly cursed at OpenStack conferences and presentations (with people talking about using CINDER + something commercial like SolidFire etc).
-
@john-nicholson said in KVM Poor Man Replication HA:
@aaronstuder said in KVM Poor Man Replication HA:
Ceph would be a better option
I think Redhat knows why they aren't using it. Its core feature set was high-speed transactional object and not block (the Block have was explicitly not production ready when they acquired it). While it is architecturally better for the block, It's dynamic failure handling is non-existent (Gluster is a bit better here because of the layering it tends to sit on and use of Local RAID which was what RedHat was packaging for HCI). Ceph is the #1 reason I've seen for OpenStack deployments to crash and burn in a disastrous way. With Careful testing, validation, lifecycle control, and prescriptive deployments it's powerful. It's just... Kinda a glass cannon, and you'll see it openly cursed at OpenStack conferences and presentations (with people talking about using CINDER + something commercial like SolidFire etc).
And CephFS still isn't production ready. Plus it requires whole separate metadata servers. RedHats hyperconverged stuff and storage stuff both use Gluster. It's easier to maintain and less prone to explode in your face.
Plus Ansible has Gluster modules
-
@john-nicholson said in KVM Poor Man Replication HA:
I deployed it for an HCI cluster (well tried). The issue was it locked read only on files on re-sync operations and certain things (which blew them up). I'm sure it's some ways, but Gluster wasn't designed for VM storage (Neither was Ceph, but it was designed to perform better at writes than gluster). The original core use cases were scaled out file services to get around limitations of scaling of XFS/EXT4 etc and do a little DRAM caching for read/metadata on nodes. It was a poor man's Isilon more than a low latency transactional filer, or general purpose block platform.
The other thing is to be careful with Gluster and quorum. It's REALLY easy to setup. It's also REALLY easy to setup something like what this guy had that was working till it didn't.
I initially tried Gluster with VM replication and found it kind of slow. What I switched to was having (some of) the VMs do replication themselves with Gluster. That way it's easy to add replicates or distribute the load instead of messing with the physical storage. It's worked pretty well so far (as exactly a poor man's Isilon).
Gluster doesn't perform super well with a ton of tiny files (at least didn't used to) and putting it on RAID 5 was not an awesome idea. Writes would have been painfully slow.
-
http://www.phoronix.com/scan.php?page=news_item&px=oVirt-4.1.5-Released
Good timing for you on this.
-
Thread resurrected !!!
I see, interesting . Regarding Ovirt + GLusterFs my update on this is that did learn glustering and it was easy to perform. I didnt apply it in production or VMs. I did use Ovirt a month ago and it was very slow web ui experience.
I should write a thread about my and Gluster.
