ZeroTier File Transfer Speed
-
Was doing some testing between 2 devices (Windows PCs at each end), each connected to a 500Mbps ISP (tested multiple times to verify) and when I transfer a file between those 2 PCs, the best I can get is around 200Mbps regardless of the direction of the file transfer. Both PCs are running Core i5, 8GB RAM and 500GB SSDs so there shouldn't be a bottleneck at either PC end.
Has anyone been able to get better than 200Mbps across ZeroTier? Just wondering if this is where it will max out.
Was planning on using this to setup remote Veeam backups.
-
How fast do you get without ZeroTier?
-
@NashBrydges said in ZeroTier File Transfer Speed:
so there shouldn't be a bottleneck at either PC end.
That's not where we'd expect the slow downs.
-
Are you testing the full speed, or the speed only inside of the tunnel?
VPNs by their very nature have to add performance overhead. First you add latency by encrypting the traffic, then the resulting payload has to be larger than the original data, then once received it has to be decrypted again which adds more latency. The added latency makes it slower for data transfers because normal transfer methods wait for responses from the other end and the added latency causes it to wait and not utilize the full bandwidth. Some streaming transfers can overcome this, but normal protocols count on the low latency.
-
@scottalanmiller said in ZeroTier File Transfer Speed:
How fast do you get without ZeroTier?
One of the PCs is at a remote location so no way to test on the same network. I've not setup any other VPN between the 2.
-
@NashBrydges said in ZeroTier File Transfer Speed:
One of the PCs is at a remote location so no way to test on the same network.
That wouldn't test the speed. You can only test the expected full speed by doing a non-VPN transfer between the two machines as they are using the same data path. Your current test is that each machine gets roughly 500Mb/s to a local server without going any distance on the Internet, that's useful when testing if your ISP gives you what you ask for, it's not useful in knowing the full data path performance between two Internet end points.
-
@scottalanmiller said in ZeroTier File Transfer Speed:
Are you testing the full speed, or the speed only inside of the tunnel?
VPNs by their very nature have to add performance overhead. First you add latency by encrypting the traffic, then the resulting payload has to be larger than the original data, then once received it has to be decrypted again which adds more latency. The added latency makes it slower for data transfers because normal transfer methods wait for responses from the other end and the added latency causes it to wait and not utilize the full bandwidth. Some streaming transfers can overcome this, but normal protocols count on the low latency.
I wanted to test how fast I could transfer files between the 2 so the short answer to your question is, yes, I'm trying to see what the max speed is that I can get.
I totally get that I'm going to face some overhead but I wasn't expecting this. And the encrypting/decrypting activity would translate to higher CPU usage but I'm definitely not seeing anything that tells me CPU is bogged down. CPU usage during file transfer is below 20%. Unless this activity is pegging a single thread meaning I wouldn't see a CPU bottleneck when looking at utilization, I'm not sure this is the issue.
I'm seeing anywhere between 5ms and 12ms either direction when pinging.
-
@NashBrydges said in ZeroTier File Transfer Speed:
I've not setup any other VPN between the 2.
No other VPN would likely be useful as all VPNs will have the same general performance. ZeroTier is basically a managed SSL VPN. We can reasonable guess that another VPN will have the same issues. The test you want to do is without a VPN. Only then can you know how much the VPN is cutting down the overall performance. I guarantee that you will find that your direct transfer doesn't come close to 500Mb/s.
Right now you are blaming the VPN for cutting down the performance, but as you don't know what the original performance is, it's impossible to know how much impact the VPN has.
-
@NashBrydges said in ZeroTier File Transfer Speed:
I wanted to test how fast I could transfer files between the 2 so the short answer to your question is, yes, I'm trying to see what the max speed is that I can get.
That test can only be done by a point to point, non-VPN test.
-
There could be a router between your two locations on the internet that is the source of the bottleneck.
This is why Scott wants you to use a non-VPN connection to make a speed test.
-
@NashBrydges said in ZeroTier File Transfer Speed:
I totally get that I'm going to face some overhead but I wasn't expecting this. And the encrypting/decrypting activity would translate to higher CPU usage but I'm definitely not seeing anything that tells me CPU is bogged down. CPU usage during file transfer is below 20%. Unless this activity is pegging a single thread meaning I wouldn't see a CPU bottleneck when looking at utilization, I'm not sure this is the issue.
I didn't say CPU bottleneck, I said added latency. Not the same thing. Latency is a guarantee, no amount of the CPU not being pegged over a period of time tells would suggest that there is no latency. Especially as CPU load is measured over a period of seconds. The VPN is absolutely pegging the CPU, all transactions do, the question is for how long. If the VPN did it for enough time to hit 100% for any reasonable measurement period, your speed would be in Kb/s, not Mb/s.
-
@NashBrydges said in ZeroTier File Transfer Speed:
I'm seeing anywhere between 5ms and 12ms either direction when pinging.
What's the latency difference inside and outside of the tunnel? But keep in mind, the more data that you transfer, the larger the latency will be. A straight ping is slightly misleading.
-
@Dashrender said in ZeroTier File Transfer Speed:
There could be a router between your two locations on the internet that is the source of the bottleneck.
There are guaranteed to be many, in fact. All of them potential contribute bottlenecks to bandwidth, all have to add latency.
-
200 Mbps is not bad. I'd be happy with that.
You'll probably find out that the speed will vary a lot from day to day and hour to hour.Best performance will be when you use the same ISP and are physically close.
It's also possible that the ISP will throttle your connection automatically depending on what port/protocol it sees. If so then running on another port sometime gets around that. -
@Pete-S said in ZeroTier File Transfer Speed:
200 Mbps is not bad. I'd be happy with that.
You'll probably find out that the speed will vary a lot from day to day and hour to hour.Best performance will be when you use the same ISP and are physically close.
It's also possible that the ISP will throttle your connection automatically depending on what port/protocol it sees. If so then running on another port sometime gets around that.That's what I was thinking. Getting 200Mb/s on a VPN over the public Internet between points of any speed is pretty amazing.
-
Hmmm...maybe my expectations were too high. I transferred a 1GB test file from the remote PC to my Nextcloud install on Vultr and got around 200Mbps as well.
-
@NashBrydges said in ZeroTier File Transfer Speed:
Hmmm...maybe my expectations were too high. I transferred a 1GB test file from the remote PC to my Nextcloud install on Vultr and got around 200Mbps as well.
Assuming you used an SSL cert, then you still have the VPN overhead in that situation. but yeah, likely you are reaching the limit of the connection via the internet.
-
@NashBrydges said in ZeroTier File Transfer Speed:
Hmmm...maybe my expectations were too high. I transferred a 1GB test file from the remote PC to my Nextcloud install on Vultr and got around 200Mbps as well.
Nextcloud by default would use SSL, so a VPN connection for all intents and purposes, so a good comparison of another VPN situation.
-
Check it with
SpeedTest.netand see what you get? -
@dafyre said in ZeroTier File Transfer Speed:
Check it with
SpeedTest.netand see what you get?that's where he started, that is useless. That's what led him astray in the first place because it shows the speed of the WAN without showing the speed of the path.
