How to let only customers download files with wget/curl?
-
@scottalanmiller said in How to let only customers download files with wget/curl?:
Not sure if this works for you as this falls under the "not solving your problem but just proposing something else" category, but we deal with this commonly and solve it by just using SSH/SFTP. It's not wget/curl, but it's super simple and standard.
Yes, that could also work but I'm looking for a https solution so you do partial downloads, have scripts serve the data and what not.
For some reason it also seems like file transfer over an ssh session is always slow at least when were talking lots of data.
-
@Pete-S said in How to let only customers download files with wget/curl?:
For some reason it also seems like file transfer over an ssh session is always slow at least when were talking lots of data.
Even with SFTP or RSYNC? It might not be HTTP fast, but it's usually pretty decent.
-
Why not force https, then allow them to wget/curl with basic authentication using something they already know like Custer number and company name (for example). That can always be logged and tracked, alerted on if it's abused.
-
If you did something like host the files in an Azure Storage Account then they could download using an SAS token that's passed in as a request parameter
-
@Obsolesce said in How to let only customers download files with wget/curl?:
Why not force https, then allow them to wget/curl with basic authentication using something they already know like Custer number and company name (for example). That can always be logged and tracked, alerted on if it's abused.
This is what I have been doing in my password manager. Each request is logged with the username and the endpoint.
-
@Pete-S said in How to let only customers download files with wget/curl?:
have scripts serve the data and what not.
Wait are you talking about CGI scripts?
-
@Obsolesce said in How to let only customers download files with wget/curl?:
Why not force https, then allow them to wget/curl with basic authentication using something they already know like Custer number and company name (for example). That can always be logged and tracked, alerted on if it's abused.
It would work but I prefer if I could get SSL client certificates up and running. You'd have to install the certificate once but then you're authorized to access "your" files on the web server. The webserver can log and track you and decide what you are allowed to access because of the client certificate.
-
@stacksofplates said in How to let only customers download files with wget/curl?:
@Pete-S said in How to let only customers download files with wget/curl?:
have scripts serve the data and what not.
Wait are you talking about CGI scripts?
Yes, that's a possibility when you are using a webserver, instead of ssh.
If you access a file over ssh, AFAIK the file is a static file and it is what it is.
If you however access a file over https, you can have a script on the webserver delivering you the file and you can send parameters to it. For instance :
wget -o install.sh "https://xyz.com/my_special_install_script.py?os=CentOS7&special=2&customer=2432"You just have a gazilion options when you connect over a webserver.
-
@Pete-S said in How to let only customers download files with wget/curl?:
@stacksofplates said in How to let only customers download files with wget/curl?:
@Pete-S said in How to let only customers download files with wget/curl?:
have scripts serve the data and what not.
Wait are you talking about CGI scripts?
Yes, that's a possibility when you are using a webserver, instead of ssh.
If you access a file over ssh, AFAIK the file is a static file and it is what it is.
If you however access a file over https, you can have a script on the webserver delivering you the file and you can send parameters to it. For instance :
wget -o install.sh "https://xyz.com/my_special_install_script.py?os=CentOS7&special=2&customer=2432"You just have a gazilion options when you connect over a webserver.
Yeah I thought you were providing files through just a default webserver. So while CGI isn't insecure by itself, you have a ton of work in securing the scripts you create. You might be better off just writing a small API to hand off the info instead of trying to properly secure CGI scripts.
Another option is a serverless function leveraging the providers authentication to serve the files up.
Here's an example from GCP where you can just check require authentication using their IAM.
-
@stacksofplates said in How to let only customers download files with wget/curl?:
@Pete-S said in How to let only customers download files with wget/curl?:
@stacksofplates said in How to let only customers download files with wget/curl?:
@Pete-S said in How to let only customers download files with wget/curl?:
have scripts serve the data and what not.
Wait are you talking about CGI scripts?
Yes, that's a possibility when you are using a webserver, instead of ssh.
If you access a file over ssh, AFAIK the file is a static file and it is what it is.
If you however access a file over https, you can have a script on the webserver delivering you the file and you can send parameters to it. For instance :
wget -o install.sh "https://xyz.com/my_special_install_script.py?os=CentOS7&special=2&customer=2432"You just have a gazilion options when you connect over a webserver.
Yeah I thought you were providing files through just a default webserver. So while CGI isn't insecure by itself, you have a ton of work in securing the scripts you create. You might be better off just writing a small API to hand off the info instead of trying to properly secure CGI scripts.
Another option is a serverless function leveraging the providers authentication to serve the files up.
Here's an example from GCP where you can just check require authentication using their IAM.
Thanks, I had the intention of letting the webserver authenticate and in most cases provide a static file directly or when needed invoke a script that will provide dynamic content.
I haven't checked nginx yet but apache can check client SSL certificates easily.
