ZeroTier Site-To-Site

lionelb

@dafyre Problem with this config : we lose 90% (or more) of the flow. It's not nothing !

Iperf from a ZT client in Azure to a ZT router on my datacenter (1 GB)
[ 4] 0.00-10.00 sec 112 MBytes 94.4 Mbits/sec sender
[ 4] 0.00-10.00 sec 112 MBytes 94.4 Mbits/sec receiver

Iperf from a ZT client in Azure to a lan machine via the same ZT router my datacenter (1 GB)
[ 4] 0.00-10.00 sec 112 MBytes 9.5 Mbits/sec sender
[ 4] 0.00-10.00 sec 112 MBytes 9.4 Mbits/sec receiver

My ZT routers are Debian on ESX VM

Or can be the problem?

JasGot

Great write up! Thanks! I looked at ZeroTier a while ago, and because I couldn't get a site-to-site going in under ten minutes, I moved on.

My only (professional) need for VPNs these days is site to site.

I'll give it another go and follow your write up.

Thanks again!

dafyre

@lionelb said in ZeroTier Site-To-Site:

@dafyre Problem with this config : we lose 90% (or more) of the flow. It's not nothing !

Iperf from a ZT client in Azure to a ZT router on my datacenter (1 GB)
[ 4] 0.00-10.00 sec 112 MBytes 94.4 Mbits/sec sender
[ 4] 0.00-10.00 sec 112 MBytes 94.4 Mbits/sec receiver

Iperf from a ZT client in Azure to a lan machine via the same ZT router my datacenter (1 GB)
[ 4] 0.00-10.00 sec 112 MBytes 9.5 Mbits/sec sender
[ 4] 0.00-10.00 sec 112 MBytes 9.4 Mbits/sec receiver

My ZT routers are Debian on ESX VM

Or can be the problem?

@lionelb said in ZeroTier Site-To-Site:

@dafyre Problem with this config : we lose 90% (or more) of the flow. It's not nothing !

Iperf from a ZT client in Azure to a ZT router on my datacenter (1 GB)
[ 4] 0.00-10.00 sec 112 MBytes 94.4 Mbits/sec sender
[ 4] 0.00-10.00 sec 112 MBytes 94.4 Mbits/sec receiver

Iperf from a ZT client in Azure to a lan machine via the same ZT router my datacenter (1 GB)
[ 4] 0.00-10.00 sec 112 MBytes 9.5 Mbits/sec sender
[ 4] 0.00-10.00 sec 112 MBytes 9.4 Mbits/sec receiver

My ZT routers are Debian on ESX VM

Or can be the problem?

@lionelb -- What happens if you go from a client in your LAN to a client in your Azure setup?

You could also try to tracert from the Azure Client to your LAN client and make sure it's not doing anything strange.

dafyre

@JasGot said in ZeroTier Site-To-Site:

Great write up! Thanks! I looked at ZeroTier a while ago, and because I couldn't get a site-to-site going in under ten minutes, I moved on.

My only (professional) need for VPNs these days is site to site.

I'll give it another go and follow your write up.

Thanks again!

If you have any questions, do let me know!

This is the set up that I am still using today.

lionelb

@dafyre I think I found the problem, my VM Debian 9 which serves as router had only 1 GB of RAM and 1 vCPU, I made x 2 and it is much better !

Here we see the difference before and after ...

dafyre

@lionelb said in ZeroTier Site-To-Site:

@dafyre I think I found the problem, my VM Debian 9 which serves as router had only 1 GB of RAM and 1 vCPU, I made x 2 and it is much better !

Here we see the difference before and after ...

Thanks for the heads up. I made a note of that in Step 2 for making the Linux router VMs.

lionelb

@dafyre And dont use Debian Buster (10), prefear Jessie (8) or Stretch (9) max at this day

dafyre

@lionelb said in ZeroTier Site-To-Site:

@dafyre And dont use Debian Buster (10), prefear Jessie (8) or Stretch (9) max at this day

I'd recommend always using the current release, so you get security updates and such. Has Buster been released yet?

black3dynamite

@dafyre said in ZeroTier Site-To-Site:

@lionelb said in ZeroTier Site-To-Site:

@dafyre And dont use Debian Buster (10), prefear Jessie (8) or Stretch (9) max at this day

I'd recommend always using the current release, so you get security updates and such. Has Buster been released yet?

Its been out for awhile now.

dafyre

@black3dynamite said in ZeroTier Site-To-Site:

@dafyre said in ZeroTier Site-To-Site:

@lionelb said in ZeroTier Site-To-Site:

@dafyre And dont use Debian Buster (10), prefear Jessie (8) or Stretch (9) max at this day

I'd recommend always using the current release, so you get security updates and such. Has Buster been released yet?

Its been out for awhile now.

Thanks. The last month or so, I have time to eat, breathe, work, sleep, and help kiddo with his homework, lol... and work has been stupid busy for the last 3 or 4 months and I have no idea why.

lionelb

@dafyre My small addition in french : https://www.canaletto.fr/post/zerotier-site-to-site

VoIP_n00b

This post is deleted!

mukky

Bro @dafyre,
You make my life much easier...
Thank you !!

After soo much hassle to achieved opnsense site2site, i found this posting solve the problems with 2 essential modification as follows:

Two essential step:

Enable IP_Forward:
in free BSD we have to edit /etc/defaults/rc.conf
change from gateway_enable="NO" to gateway_enable="YES"
Set up the Site Routes at the Routers for Site A and Site B
it has configured and implemented in opnsense router section

@dafyre, since no body cover this on opnsense, I think it will wonderful, if you could made this video on youtube as well

Good Luck !!

mukky

@mukky said in ZeroTier Site-To-Site:

Bro @dafyre,
You make my life much easier...
Thank you !!

After soo much hassle to achieved opnsense site2site, i found this posting solve the problems with 2 essential modification as follows:

Two essential step:

Enable IP_Forward:
in free BSD we have to edit /etc/defaults/rc.conf
change from gateway_enable="NO" to gateway_enable="YES"

Set up the Site Routes at the Routers for Site A and Site B
it has configured and implemented in opnsense router section

@dafyre, since no body cover this on opnsense, I think it will wonderful, if you could made this video on youtube as well

Good Luck !!

I was struggeling for a month to figure it out, not much info on internet nor tutorial regarding zerotier for site2site. Eventually i succeed to make it work.

The key point to setting on opnsense are:

you have to install zerotier plugin

you have to make your own network on your zerotier account

you have to enable zerotier on your opnsense and adding zerotier connection in it to join your own network.

you have to assign network for zerotier - dont forget to "check" Enable Interface and Prevent interface removal. Also you have to put static ip with is the same ip address as shown on your zerotier joined network.

you have to put firewall rule for zerotier to accept any incoming traffic

you have to put firewall rule for WAN/ISP to accept any incoming traffic from specific source "Ztier.net"

in some cases it requires booting/restart your opnsense to take effect.

setting above will allow any incoming connection from any remote device via zerotier towards your opnsense ip address. (Ref: opnsense ip address = ip address of WAN/ISP). In result, you can remote access your opnsense via laptop from another city / ISP (laptop must have zerotier connection and joint the same network too). On your laptop you will be able to access your opnsense by its ip address assigned by zerotier.

in the case, for example, there is a NAS behind the opnsense that you want to access remotely,....... then you only have to open your zerotier account and put a route rule there

assumed:

your NAS local ip address: 192.168.5.10

NAS local Network on opnsense: LAN-1

your opnsense ip address assigned by Zerotier: 10.188.22.10

then you have to put firewall rule for LAN-1 to accept any incoming traffic from specific source "Ztier.net"

then you have to add "route" on your zerotier account dashboard:

192.168.5.10/32 via 10.188.22.10

in result from remote laptop you can remote access:

a. opnsense by pointing to 10.188.22.10

b. NAS by pointing to 192.168.5.10

(laptop must have zerotier connection and joint the same network too)

Thats it, good luck !