Networking and 1U Colocation
-
We used to use dirt cheap colo in Scranton. It was cheap because it was old, beat up racks, in a nasty falling down old warehouse, with no services, no redundancy on the ISPs, constant outages, and most of their "servers" were $50 old desktops with open cases and fans blowing on them.
-
@scottalanmiller How fast of a connection do you need to run a site like ML? Could your $40 Vultr LAMP server run well on a 10Mbit connection?
-
@bnrstnr said in Networking and 1U Colocation:
@scottalanmiller How fast of a connection do you need to run a site like ML? Could your $40 Vultr LAMP server run well on a 10Mbit connection?
ML needs power, we don't really measure the bandwidth needs. It might be fine on 10Mb/s, but we do some serious traffic, so it would definitely be pushing it.
Our $40 LAMP server could run on 10Mb/s.
But neither of those could run in a non-production facility, so that they CAN use 10Mb/s individually isn't really a factor. They would need to use them altogether and they'd need a production level facility. None of which they could get.
-
The colo my stuff is in has 10/10 metered. I often spike 100 mbps but never deal with overages because the usage still comes out < 10/10.
-
@jaredbusch said in Networking and 1U Colocation:
The colo my stuff is in has 10/10 metered. I often spike 100 mbps but never deal with overages because the usage still comes out < 10/10.
That's a bit different, that's just a pricing model, rather than a 10Mb/s limit.
-
This is the one I currently use, but $20/month because I got it when it was on special:
-
@tim_g said in Networking and 1U Colocation:
This is the one I currently use, but $20/month because I got it when it was on special:
That's not colo, though.
-
@scottalanmiller said in Networking and 1U Colocation:
@tim_g said in Networking and 1U Colocation:
This is the one I currently use, but $20/month because I got it when it was on special:
That's not colo, though.
Yes, he was clear stating he currently used them for VPS. Did you miss that?
-
@jaredbusch said in Networking and 1U Colocation:
@scottalanmiller said in Networking and 1U Colocation:
@tim_g said in Networking and 1U Colocation:
This is the one I currently use, but $20/month because I got it when it was on special:
That's not colo, though.
Yes, he was clear stating he currently used them for VPS. Did you miss that?
I saw it earlier, but then was confused here.
-
I'm trying to think through how I'd configure my network using a VM on my KVM host as my router/firewall. This is the first draft of how I see this working.
The networks with which I've worked in the past have a firewall that would have one interface with a public IP (100.100.100.2/30) and one with a private IP (192.168.1.1/24), which connects to a switch, which connects to everything else. NAT would be used to direct traffic to specific hosts, such as NATing inbound traffic for 100.100.100.2:80 to a webserver at 192.168.1.10.
I'm trying to wrap my head around how to make that concept work in an virtualized environment where I have a single hypervisor and no networking appliance. I'd obviously have a virtual switch which wouldn't have any connection to a physical NIC on the host for the guest VMs (the 192.168.1.0/24 network). I'd have another virtual switch which would have a connection to the physical NIC. The firewall VM would have a connection to each virtual switch.
The question becomes what NIC receives the public IP address I'll be assigned. My first thought was the interface of the firewall VM that's attached to the virtual switch that's attached to the NIC of the physical host. However, that seems problematic, as at the moment, I don't know how a packet destined for 100.100.100.2 coming in on eth0 of the host would be forwarded to the virtual switch.
There are other questions; however, I think this has to be addressed first.
-
With a /30 it is hard because you have to setup your host without a valid IP on the colo network.
So you will need a private management network to access the host remotely.
You will have to rely on the colo's KVM for console access to your dom0 to fix anything if it breaks.
You should easily be able to configure this ahead of time.
-
This will get into how you want to configure your KVM host's networking in the first place.
I don't have network connectivity from my KVM host in my home lab to my guests because I have full access to the host's main IP and never bothered to set anything else up.
-
@eddiejennings said in Networking and 1U Colocation:
I'm trying to think through how I'd configure my network using a VM on my KVM host as my router/firewall. This is the first draft of how I see this working.
The networks with which I've worked in the past have a firewall that would have one interface with a public IP (100.100.100.2/30) and one with a private IP (192.168.1.1/24), which connects to a switch, which connects to everything else. NAT would be used to direct traffic to specific hosts, such as NATing inbound traffic for 100.100.100.2:80 to a webserver at 192.168.1.10.
I'm trying to wrap my head around how to make that concept work in an virtualized environment where I have a single hypervisor and no networking appliance. I'd obviously have a virtual switch which wouldn't have any connection to a physical NIC on the host for the guest VMs (the 192.168.1.0/24 network). I'd have another virtual switch which would have a connection to the physical NIC. The firewall VM would have a connection to each virtual switch.
The question becomes what NIC receives the public IP address I'll be assigned. My first thought was the interface of the firewall VM that's attached to the virtual switch that's attached to the NIC of the physical host. However, that seems problematic, as at the moment, I don't know how a packet destined for 100.100.100.2 coming in on eth0 of the host would be forwarded to the virtual switch.
There are other questions; however, I think this has to be addressed first.
Maybe try this, lets see if I can explain myself well.
Attach your physical interface to a linux-bridge (lets call it br0) and then assign the br0 interface only to your firewall vm. Set the static address 100.100.100.2 on the vms interface this will be your wan port .
Create a private network and set it to isolated, it will create a virbrX interface in your host. Add an interface to your vm inside the isolated private network (using virbrX), this will be your firewalls lan interface.
Your virtual network gives you internal and host only routing, so you will have access to your guest through this ip range.
Any new vm you create add it interface to be part of your virtual network and set their gateway to your firewall-vms lan port.
I believe that should do what you want.
-
@romo He does not even need a bridge. Simply no IP on his eth0.
He will have a java based KVM available for console from his colo if shit really breaks.
Here in my example I have an IP on eno1 just because it is local to me and i am lazy. He should have no IP here.
I have a team, he will not with only a single port from the colo. He could have a team still with one memeber not plugged in, but meh.
When he makes the pfSense VM, he should assign it to his eth0 (team0 in my example) as macvtap.
In bridged mode. As the note states, there will not be any guest communication to the host on this network, which is just fine for security of the router VMs public IP network.
By default the LVM setup create this private network. I suggest he use this on his host and a single guest VM (not his pfSense VM) to manage the host.
I suggested he make another private network that he will assign to the LAN port of the pfSense VM and then as the only network port for all of the guest VMs.
-
@jaredbusch I am so used to creating my bridges manually I keep forgetting macvtap creates them for you.
-
I had to do something a little different as I rent a server that I'll never have physical access to. So this is on an Ubuntu Server base (that always leaves a bad taste in my mouth, sorry.)
eth0 is all I had to work with:
So I also made a virtual interface:
Virtual Interface that is the public facing side:
Finally, the private side:
Now, if you ask me (you didn't, I'll tell you anyway), this way of configuring things kinda sucks. I'm limited to 100mb/s throughput by something along the line that I haven't figured out yet.... I've got stories about these already, I don't need more!
-
@jaredbusch said in Networking and 1U Colocation:
I want to make sure my understanding of your suggestion is correct.
The FirewallVM will have one NIC connected to eth0 via macvtap, which would be assigned 100.100.100.2/30. The FirewallVM will have a second NIC attached to what you called the priv0 virtual network, which serves as the interface to the rest of the VMs in the LAN. For managing the host itself, I'll have another VM attached to the default virtual network.
-
I'm thinking through this topology a bit further:
- Eth0 on host passes traffic via macvtap to FirewallVM (100.100.100.2/30).
- Isolated network connects FirewallVM to the rest of the guest VM.
- Isolated network connects one of the guest VMs to the host VM.
To manage my host over the Internet, I'd have a NAT rule that would direct SSH traffic to VM in item three. Once connected to that VM, then I'd SSH from there to my host -- basically making that guest VM jump box.
-
@eddiejennings said in Networking and 1U Colocation:
I'm thinking through this topology a bit further:
- Eth0 on host passes traffic via macvtap to FirewallVM (100.100.100.2/30).
- Isolated network connects FirewallVM to the rest of the guest VM.
- Isolated network connects one of the guest VMs to the host VM.
To manage my host over the Internet, I'd have a NAT rule that would direct SSH traffic to VM in item three. Once connected to that VM, then I'd SSH from there to my host -- basically making that guest VM jump box.
VyOS has ssh built in.... just saying.
-
@travisdh1 said in Networking and 1U Colocation:
@eddiejennings said in Networking and 1U Colocation:
I'm thinking through this topology a bit further:
- Eth0 on host passes traffic via macvtap to FirewallVM (100.100.100.2/30).
- Isolated network connects FirewallVM to the rest of the guest VM.
- Isolated network connects one of the guest VMs to the host VM.
To manage my host over the Internet, I'd have a NAT rule that would direct SSH traffic to VM in item three. Once connected to that VM, then I'd SSH from there to my host -- basically making that guest VM jump box.
VyOS has ssh built in.... just saying.
Umm WTF? Open SSH to the world? On your router? Fuck that.
