Alternative to FTP
-
@nadnerB said in Alternative to FTP:
Just because it's needed ASAP, doesn't mean it shouldn't be done properly
I'd prefer micro SD and carrier pigeon over FTP.Seriously though "Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway".
Have you ruled out driving it there? It' meets two of your criteria: simple, and alternative.I know Internet outside of cities is still not stellar since we NBN'd, but bandwidth is still quite expensive so not everyone can move gigabytes of data in a timely fashion.
As for a technical solution using NextCloud, OneDrive, Dropbox etc, is what I'd be looking at.
It's simpler for the users at both ends.This is a great point - you're talking about uploading GB's of data from their customers to their servers - I hope they have a great internet pipe for that.
The sending side will also likely be the primary bottleneck point, as most ISPs are asymetrical, many topping out at 20 mb/s. -
I agree that the "easiest" is to use cloud storage such as OneDrive or something like that.
And have the local server sync to the cloud storage. So when a client uploads something from their computer to the cloud storage it will automatically be downloaded to the local server.
-
@Pete-S said in Alternative to FTP:
I agree that the "easiest" is to use cloud storage such as OneDrive or something like that.
And have the local server sync to the cloud storage. So when a client uploads something from their computer to the cloud storage it will automatically be downloaded to the local server.
This doesn't provide the requested, ASAP onsite on the server component in the OP.
That said - I'd be looking at - why does it need to be local? any way to have this potentially hosted and do the work on the file in that hosted location? You're much more likely able to get great large bandwidth at low cost into a hosted solution (or colo) and not affect the people on your local network.
-
@Dashrender said in Alternative to FTP:
This doesn't provide the requested, ASAP onsite on the server component in the OP.
It kind of depends on the definition of asap right?
The customer uploads and it's immediately synced to the local server. That's asap. No second step is required.
I don't know if the sync software will wait until it's completely uploaded or start to download blocks of a partial file. I guess that would be up to the sync software.
-
@Pete-S said in Alternative to FTP:
@Dashrender said in Alternative to FTP:
This doesn't provide the requested, ASAP onsite on the server component in the OP.
It kind of depends on the definition of asap right?
The customer uploads and it's immediately synced to the local server. That's asap. No second step is required.
I don't know if the sync software will wait until it's completely uploaded or start to download blocks of a partial file. I guess that would be up to the sync software.
And asap might not be so fast if the client has to install software and can't get it to work or the software can't handle the upload being interrupted midway. Or if the clients networks slows down to a crawl for hours because it uses too much bandwidth.
I'd like to know how big are the files or are there multiple files per "transfer". What kind of bandwidth does the server has to the internet? How much of that bandwidth can it use? And the same for the typical client. And how many clients and how often will this happen? Will the same client transfer files over and over or will mostly be a one-time thing. And how fast is asap?
-
@Dashrender said in Alternative to FTP:
That said - I'd be looking at - why does it need to be local? any way to have this potentially hosted and do the work on the file in that hosted location? You're much more likely able to get great large bandwidth at low cost into a hosted solution (or colo) and not affect the people on your local network.
It's possible that its video files, raw image files or something of that nature that they want to start working with immediately. And that is going to be local files.
If it's the same clients doing the same kind of file transfer on a regular basis and you want the minimum amount of time from the start of the transfer to the file is local, I agree with @scottalanmiller that sftp would be the easiest. Have the clients install Filezilla and use that to transfer the files.
-
@Pete-S said in Alternative to FTP:
@Dashrender said in Alternative to FTP:
This doesn't provide the requested, ASAP onsite on the server component in the OP.
It kind of depends on the definition of asap right?
The customer uploads and it's immediately synced to the local server. That's asap. No second step is required.
I don't know if the sync software will wait until it's completely uploaded or start to download blocks of a partial file. I guess that would be up to the sync software.
Sure, not a second step by the company - but the system has two steps in it - 1)upload to cloud service 2) download/sync to local server.
this makes it take more time to transfer. -
Syncthing could be an option here
-
@flaxking said in Alternative to FTP:
Syncthing could be an option here
Definitely a possibility for something that happens all of the time.
-
Thanks everyone for the input.
It's medical data that needs to be looked at by medical software which resides on a local server.
I had considered syncing using Onedrive, but thought I'd ask and see what else popped up.
Driving to site is not an option unfortunately, this could be a nationwide thing.
As I type I'm thinking the Filezilla solution may be a good one. I can supply the senders with info on how to configure FZ and have as a step of the operation, for them to contact the receiver once the upload has completed.
This will be an unsupervised operation, so with anything that syncs, there is the potential for people to grab the upload before the sync has completed.
-
@Dashrender said in Alternative to FTP:
@Pete-S said in Alternative to FTP:
I agree that the "easiest" is to use cloud storage such as OneDrive or something like that.
And have the local server sync to the cloud storage. So when a client uploads something from their computer to the cloud storage it will automatically be downloaded to the local server.
This doesn't provide the requested, ASAP onsite on the server component in the OP.
That said - I'd be looking at - why does it need to be local? any way to have this potentially hosted and do the work on the file in that hosted location? You're much more likely able to get great large bandwidth at low cost into a hosted solution (or colo) and not affect the people on your local network.
A hosted solution is a good thought. I like the 'high bandwidth' idea.
Two things for me to consider:- Storage of medical data offsite
- Cost of installation of the medical software onto the host
- Can the medical software be installed on the host?
-
@siringo you might want to rethink Filezilla, last I heard it the installer was bundling some sketchy stuff with it. I just use WinSCP when needed.
-
Also if you go with FTPS instead of SFTP you have to deal with UTM/Next Gen Firewalls messing with the traffic on the client side all the time.
-
@siringo said in Alternative to FTP:
Thanks everyone for the input.
It's medical data that needs to be looked at by medical software which resides on a local server.
I had considered syncing using Onedrive, but thought I'd ask and see what else popped up.
Driving to site is not an option unfortunately, this could be a nationwide thing.
As I type I'm thinking the Filezilla solution may be a good one. I can supply the senders with info on how to configure FZ and have as a step of the operation, for them to contact the receiver once the upload has completed.
This will be an unsupervised operation, so with anything that syncs, there is the potential for people to grab the upload before the sync has completed.
Another sftp option is to use
sshfs(ssh filesystem).We use it on windows and linux. It allows you to mount a remote file system, like a network share if you will.
Then you can just drop any file there.The users don't even realize what going on behind the scenes. All they need to do is the same as when they copy a file in Windows.
https://mangolassi.it/topic/21159/how-to-mount-remote-filesystem-over-ssh-both-windows-linux
-
@flaxking thanks for your help, it's appreciated.
-
@Pete-S I had a very quick look over your ML post on this so nothing has sunk in, but how do external people access a share within a local network?
Do I have to set them up with a uname & pwd ???
-
@siringo said in Alternative to FTP:
@Pete-S I had a very quick look over your ML post on this so nothing has sunk in, but how do external people access a share within a local network?
Do I have to set them up with a uname & pwd ???
You'd have to publish the server to the internet - and yes create each person a username and password.
you also need to make sure they can't get to other directories on the server besides their own. -
@siringo said in Alternative to FTP:
@Pete-S I had a very quick look over your ML post on this so nothing has sunk in, but how do external people access a share within a local network?
Do I have to set them up with a uname & pwd ???
It's using SSH under the hood, so whatever mechanism you'd use to make a remote SSH user.
-
@Dashrender said in Alternative to FTP:
@siringo said in Alternative to FTP:
@Pete-S I had a very quick look over your ML post on this so nothing has sunk in, but how do external people access a share within a local network?
Do I have to set them up with a uname & pwd ???
You'd have to publish the server to the internet - and yes create each person a username and password.
you also need to make sure they can't get to other directories on the server besides their own.Yes in it's simplest form create a user account & password for each client that needs access. When the client creates the sshfs share they enter the servername, username and password. Then they can upload their files - and download too if you want to send files the other direction.
Your client, that has the server, can also use a sshfs share. You can create another user account (or several) for that. The user account just need to have read permission to the files that their clients have uploaded.
-
@Pete-S thanks for pointing this out, it does look good, I'd like to test it out.
I'm suspecting that since it's a file copy operation from Windows, there'll be nothing like auto-reconnect, retry on reconnect etc etc?
