UBNT EdgeRouter site to site VPN routes?

art_of_shred

vtun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 100
    link/none
    inet 10.99.99.2 peer 10.99.99.1/32 scope global vtun0
       valid_lft forever preferred_lft forever

    RX:  bytes    packets     errors    dropped    overrun      mcast
             0          0          0          0          0          0
    TX:  bytes    packets     errors    dropped    carrier collisions
       3231942      44734          0          0          0          0


show interfaces openvpn vtun0
vtun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state                        UNKNOWN group default qlen 100
    link/none
    inet 10.99.99.1 peer x.x.x.218/32 scope global vtun0
       valid_lft forever preferred_lft forever

    RX:  bytes    packets     errors    dropped    overrun      mcast
             0          0          0          0          0          0
    TX:  bytes    packets     errors    dropped    carrier collisions
       6962847      28715          0          0          0          0

art_of_shred

I see one of the peers is using another IP... but I don't know how to change that.

art_of_shred

Ok, so the x.x.x.218 IP is the WAN IP of the opposite reouter, instead of using the openvpn IP.

JaredBusch

#if not in config already 
configure

#then remove all the stuff.
delete interfaces openvpn vtun0
delete protocols static interface-route 192.168.1.0/24
delete protocols static interface-route 192.168.2.0/23

#recreate it
set interfaces openvpn vtun0 local-address 10.99.99.1 
set interfaces openvpn vtun0 local-port 1194
set interfaces openvpn vtun0 mode site-to-site
set interfaces openvpn vtun0 openvpn-option --float
set interfaces openvpn vtun0 openvpn-option "--ping 10"
set interfaces openvpn vtun0 openvpn-option "--ping-restart 20"
set interfaces openvpn vtun0 openvpn-option --ping-timer-rem
set interfaces openvpn vtun0 openvpn-option --persist-tun
set interfaces openvpn vtun0 openvpn-option --persist-key
set interfaces openvpn vtun0 openvpn-option "--user nobody"
set interfaces openvpn vtun0 openvpn-option "--group nogroup"
set interfaces openvpn vtun0 remote-address 10.99.99.2
set interfaces openvpn vtun0 remote-host x.x.x.218
set interfaces openvpn vtun0 remote-port 1194
set interfaces openvpn vtun0 shared-secret-key-file /config/auth/secret
set protocols static interface-route 192.168.1.0/24 next-hop-interface vtun0
commit

#if works
save
exit

#other side

#if not in config already 
configure

#then remove all the stuff.
delete interfaces openvpn vtun0
delete protocols static interface-route 192.168.1.0/24
delete protocols static interface-route 192.168.2.0/23

#recreate it
set interfaces openvpn vtun0 local-address 10.99.99.2
set interfaces openvpn vtun0 local-port 1194
set interfaces openvpn vtun0 mode site-to-site
set interfaces openvpn vtun0 openvpn-option --float
set interfaces openvpn vtun0 openvpn-option "--ping 10"
set interfaces openvpn vtun0 openvpn-option "--ping-restart 20"
set interfaces openvpn vtun0 openvpn-option --ping-timer-rem
set interfaces openvpn vtun0 openvpn-option --persist-tun
set interfaces openvpn vtun0 openvpn-option --persist-key
set interfaces openvpn vtun0 openvpn-option "--user nobody"
set interfaces openvpn vtun0 openvpn-option "--group nogroup"
set interfaces openvpn vtun0 remote-address 10.99.99.1
set interfaces openvpn vtun0 remote-host x.x.x.51
set interfaces openvpn vtun0 remote-port 1194
set interfaces openvpn vtun0 shared-secret-key-file /config/auth/secret
set protocols static interface-route 192.168.2.0/23 next-hop-interface vtun0

commit

#if works
save
exit

JaredBusch

if i got the static routing protocols backwards, just reverse them. They should point to the LAN on the opposite router.

That is the last line prior to each commit.

art_of_shred

Ok, so it all looks good. What would be the best test?

art_of_shred

I can't ping LAN IP's on the opposite side...

JaredBusch

@art_of_shred said in UBNT EdgeRouter site to site VPN routes?:

I can't ping LAN IP's on the opposite side...

Well if the tunnel is up, you should.

I intentionally deleted the OpenVPN interfaces just to make sure there were no firewall policies hanging around on them.

So start with the basic. is the tunnel actually up and able to pass traffic.

From router 1 ping the IP on the other end of the OpenVP tunnel.

ping 10.99.99.1 or ping 10.99.99.2 whichever is on the opposite side

nothing but the routers will be able to use these addresses. they are only for pinning up the OpenVPN tunnel

art_of_shred

Yeah, no dice.

JaredBusch

@art_of_shred said in UBNT EdgeRouter site to site VPN routes?:

Yeah, no dice.

Then the tunnel is not up. Something else was done wrong.

JaredBusch

open up 2 ssh sessions to one of the routers.

Do not go into config mode.

in one, watch the log, show log tail

in the other window, reset the OpenVPN connection reset openvpn interface vtun0

see if anything in the log is useful

coliver

Out of curiosity is there a reason to use OpenVPN over IPSEC?

art_of_shred

Jul  5 17:23:23 ubnt openvpn[3172]: Restart pause, 2 second(s)
Jul  5 17:23:25 ubnt openvpn[3172]: Re-using pre-shared static key
Jul  5 17:23:25 ubnt openvpn[3172]: Socket Buffers: R=[294912->131072] S=[294912                                  ->131072]
Jul  5 17:23:25 ubnt openvpn[3172]: Preserving previous TUN/TAP instance: vtun0
Jul  5 17:23:25 ubnt openvpn[3172]: UDPv4 link local (bound): [undef]
Jul  5 17:23:25 ubnt openvpn[3172]: UDPv4 link remote: [AF_INET]x.x.x.218:1                                  194
Jul  5 17:23:36 ubnt openvpn[3172]: event_wait : Interrupted system call (code=4                                  )
Jul  5 17:23:36 ubnt openvpn[3172]: SIGUSR1[hard,] received, process restarting
Jul  5 17:23:36 ubnt openvpn[3172]: Restart pause, 2 second(s)
Jul  5 17:23:38 ubnt openvpn[3172]: Re-using pre-shared static key
Jul  5 17:23:38 ubnt openvpn[3172]: Socket Buffers: R=[294912->131072] S=[294912                                  ->131072]
Jul  5 17:23:38 ubnt openvpn[3172]: Preserving previous TUN/TAP instance: vtun0
Jul  5 17:23:38 ubnt openvpn[3172]: UDPv4 link local (bound): [undef]
Jul  5 17:23:38 ubnt openvpn[3172]: UDPv4 link remote: [AF_INET]x.x.x.218:1                                  194
Jul  5 17:23:58 ubnt openvpn[3172]: Inactivity timeout (--ping-restart), restarting
Jul  5 17:23:58 ubnt openvpn[3172]: SIGUSR1[soft,ping-restart] received, process restarting

JaredBusch

@coliver said in UBNT EdgeRouter site to site VPN routes?:

Out of curiosity is there a reason to use OpenVPN over IPSEC?

This is not my setup, so I have no idea on their reasons.
I use OpenVPN when connecting a home router into an office for site to site because OpenVPN has long worked better (for me) with dynamic IP addresses.
I will use IPSEC for static assigned offices because you get more throughput when offloading is enabled.

art_of_shred

@JaredBusch said in UBNT EdgeRouter site to site VPN routes?:

@coliver said in UBNT EdgeRouter site to site VPN routes?:

Out of curiosity is there a reason to use OpenVPN over IPSEC?

This is not my setup, so I have no idea on their reasons.
I use OpenVPN when connecting a home router into an office for site to site because OpenVPN has long worked better (for me) with dynamic IP addresses.
I will use IPSEC for static assigned offices because you get more throughput when offloading is enabled.

I don't see any reason we couldn't/shouldn't use IPSEC.

JaredBusch

@art_of_shred said in UBNT EdgeRouter site to site VPN routes?:

@JaredBusch said in UBNT EdgeRouter site to site VPN routes?:

@coliver said in UBNT EdgeRouter site to site VPN routes?:

Out of curiosity is there a reason to use OpenVPN over IPSEC?

This is not my setup, so I have no idea on their reasons.
I use OpenVPN when connecting a home router into an office for site to site because OpenVPN has long worked better (for me) with dynamic IP addresses.
I will use IPSEC for static assigned offices because you get more throughput when offloading is enabled.

I don't see any reason we couldn't/shouldn't use IPSEC.

Run the delete commands from before, then go into the GUI and setup IPSEC.

delete blah blah
commit
save
exit

go to GUI.

art_of_shred

well, I can ping from site B to site A now, so that's good. Both sides said the configuration was applied successfully. I'm trying to ping router-to-router and it only works one way. In advanced options, the NAT exclusion is set and applied.

art_of_shred

My bad. I didn't realize that IMCP response was disabled on the site B router. I can ping across the tunnel to servers on the site B LAN. Success!

Huge thanks to @JaredBusch for all of your help today!

Mike Davis

Big thanks to @JaredBusch I was in a hurry to leave for a speaking engagement and had to hand off to @art_of_shred . It turns out part of the problem was that some of the servers on the far side had a persistent route set up so that even when we changed the gateway address from the Meraki to the EdgeRouter, they were still hitting the Meraki. Now that it's off hours I ripping through 19 servers to make sure the gateway is correct and there are no persistent routes configured that will mess things up.

scottalanmiller

@Mike-Davis Assume everything dumb that could be done has been done down there. What a mess.