Security Of Cloud Shared Links
-
@scottalanmiller said in Security Of Cloud Shared Links:
Example...
http://mirror.centos.org/centos/
That link doesn't go to a file, it goes to a directory. but what does your browser display? An HTML page automatically generated by the web server that links to all files and folders stored in that location. Follow the links to automatically generated pages with more links until you get to the file that you want. A spider following this is literally following links generated by an application making HTML pages behind the scenes to display the links to the end users (or spiders.) This is not intrinsic but is a "by convention" method of displaying static HTML folders and files and is super common to not use (the web server that NodeBB uses doesn't even have this functionality.)
If you don't automatically make those links one way or another, the spider has nothing to follow.
OK that makes sense. So if, in the case of NodeBB, it doesn't have the functionality, but there a folder called /pictures123 that has pictures in it and there are no links to it, and not auto generation - yet if you know the exact URL, you're saying Google can't find that folder? and no legal entity can?
-
Right... the HTTP command set has no listing capability at all. That's not one of the HTTP commands. All directory listings or links of any sort have to be either included in a static file or created by the web server or something that talks to the web server (like a PHP site.)
-
@scottalanmiller said in Security Of Cloud Shared Links:
@Dashrender said in Security Of Cloud Shared Links:
@scottalanmiller said in Security Of Cloud Shared Links:
Even index.html isn't an exception to this, that is autolinked as a setting in the web server, too. Many of the links are by convention.
auto linked to what?
To whatever the default is set in the web server. Go to http://ntg.co/ (really, go to it, we need the hits) and the web server is informed to go "serve up the default link." In the case of that particular site, the default is set to index.php. You set this for all web servers. If you don't set it for Apache, it has a default setting of index.html and IIS has a built in default of index.htm.
The whole it's linked is where I'm getting hung up here - I agree that those are the default locations where those services will send people - but calling them links or linked - not sure you've convinced me yet.
-
@Dashrender said in Security Of Cloud Shared Links:
@scottalanmiller said in Security Of Cloud Shared Links:
Example...
http://mirror.centos.org/centos/
That link doesn't go to a file, it goes to a directory. but what does your browser display? An HTML page automatically generated by the web server that links to all files and folders stored in that location. Follow the links to automatically generated pages with more links until you get to the file that you want. A spider following this is literally following links generated by an application making HTML pages behind the scenes to display the links to the end users (or spiders.) This is not intrinsic but is a "by convention" method of displaying static HTML folders and files and is super common to not use (the web server that NodeBB uses doesn't even have this functionality.)
If you don't automatically make those links one way or another, the spider has nothing to follow.
OK that makes sense. So if, in the case of NodeBB, it doesn't have the functionality, but there a folder called /pictures123 that has pictures in it and there are no links to it, and not auto generation - yet if you know the exact URL, you're saying Google can't find that folder? and no legal entity can?
Right, Google has no means to look for that folder or its contents.
-
@StrongBad said in Security Of Cloud Shared Links:
Right... the HTTP command set has no listing capability at all. That's not one of the HTTP commands. All directory listings or links of any sort have to be either included in a static file or created by the web server or something that talks to the web server (like a PHP site.)
OK, that was something I didn't know. Thanks.
Then what's the problem with leaving the default directories and junk behind in an IIS install - if nothing links to those things, what harm is there in them being there? I suppose their being there as a directly accessible folder and the tyranny of the default is what creates the harm. So if you know that that is a default folder, you can try to go there directly and attempt to execute something that might be there...
-
@Dashrender said in Security Of Cloud Shared Links:
@scottalanmiller said in Security Of Cloud Shared Links:
@Dashrender said in Security Of Cloud Shared Links:
@scottalanmiller said in Security Of Cloud Shared Links:
Even index.html isn't an exception to this, that is autolinked as a setting in the web server, too. Many of the links are by convention.
auto linked to what?
To whatever the default is set in the web server. Go to http://ntg.co/ (really, go to it, we need the hits) and the web server is informed to go "serve up the default link." In the case of that particular site, the default is set to index.php. You set this for all web servers. If you don't set it for Apache, it has a default setting of index.html and IIS has a built in default of index.htm.
The whole it's linked is where I'm getting hung up here - I agree that those are the default locations where those services will send people - but calling them links or linked - not sure you've convinced me yet.
It only sends people there because of the links. You can argue that index.html is not a link, but only that one case. All other resources are only available by links, there is no default to get people there.
-
@Dashrender said in Security Of Cloud Shared Links:
Then what's the problem with leaving the default directories and junk behind in an IIS install - if nothing links to those things, what harm is there in them being there? I suppose their being there as a directly accessible folder and the tyranny of the default is what creates the harm. So if you know that that is a default folder, you can try to go there directly and attempt to execute something that might be there...
Because if it is generic, then malicious users can try to access it because they know that it is commonly there. It's a bigger attack surface.
-
My example I used was that if I create a page OUTSIDE my WordPress site, there is no way anything can find it.
So if I made www.brra.com/SAM, no one would ever see it.
Well, they would now because I posted here on ML. But you get the drift.
-
@scottalanmiller said in Security Of Cloud Shared Links:
@Dashrender said in Security Of Cloud Shared Links:
@scottalanmiller said in Security Of Cloud Shared Links:
@Dashrender said in Security Of Cloud Shared Links:
@scottalanmiller said in Security Of Cloud Shared Links:
Even index.html isn't an exception to this, that is autolinked as a setting in the web server, too. Many of the links are by convention.
auto linked to what?
To whatever the default is set in the web server. Go to http://ntg.co/ (really, go to it, we need the hits) and the web server is informed to go "serve up the default link." In the case of that particular site, the default is set to index.php. You set this for all web servers. If you don't set it for Apache, it has a default setting of index.html and IIS has a built in default of index.htm.
The whole it's linked is where I'm getting hung up here - I agree that those are the default locations where those services will send people - but calling them links or linked - not sure you've convinced me yet.
It only sends people there because of the links. You can argue that index.html is not a link, but only that one case. All other resources are only available by links, there is no default to get people there.
I completely agree with the rest being links.. just the default page loaded when visiting a folder directly be it www.google.com or www.google.com\scottalenmiller
-
@BRRABill said in Security Of Cloud Shared Links:
My example I used was that if I create a page OUTSIDE my WordPress site, there is no way anything can find it.
So if I made www.brra.com/SAM, no one would ever see it.
Well, they would now because I posted here on ML. But you get the drift.
Yesterday I would have said that Google could find it.. but now with an education from Scott and strongbad - I guess not.
-
@Dashrender said
Yesterday I would have said that Google could find it.. but now with an education from Scott and strongbad - I guess not.
You and me, both.
I really thought that is how it worked. It crawled through the SITE looking for files, not looking for links.
-
@BRRABill said in Security Of Cloud Shared Links:
@Dashrender said
Yesterday I would have said that Google could find it.. but now with an education from Scott and strongbad - I guess not.
You and me, both.
I really thought that is how it worked. It crawled through the SITE looking for files, not looking for links.
Well - that's kinda semantics, but not entirely.
What I didn't know, that @StrongBad pointed out, is that the HTTP protocol has not way of displaying content of a folder itself. That those webservers that do show the folder contents do so because of a function of the web server, not a function of HTTP - and on the web server side, it can be turned off - which was something I know could happen, but I didn't know to what level it actually kept people out - sounds like it actually does a pretty damned good job. -
I didn't realize I could put items outside the realm on my site and not have them seen.
Sweet.
-
Then there's the otherside of this - the fact that there aren't that many static pages anymore. Most of the time things are generated on the fly by an application installed into the web server, such as WordPress. So even if you could search the directory, there wouldn't be anything there. instead the file is created only upon request and delivered to the end user, and not written to the directory.
-
@BRRABill said in Security Of Cloud Shared Links:
I didn't realize I could put items outside the realm on my site and not have them seen.
Sweet.
Only not seen as long as someone doesn't guess the direct path - but now we're back to guessing the path to the above mentioned sharing files - if someone guesses it right, they get right in, but what are the chances? 1 in 10^42?
-
@Dashrender said
Only not seen as long as someone doesn't guess the direct path - but now we're back to guessing the path to the above mentioned sharing files - if someone guesses it right, they get right in, but what are the chances? 1 in 10^42?
Yeah, almost impossible.
-
@Dashrender said in Security Of Cloud Shared Links:
I completely agree with the rest being links.. just the default page loaded when visiting a folder directly be it www.google.com or www.google.com\scottalenmiller
Right, that one is a link because it is listed in DNS. So still a link, just not a generated one
So http://ntg.co/ is still a link, just one linked from DNS.
-
@Dashrender said in Security Of Cloud Shared Links:
Then there's the otherside of this - the fact that there aren't that many static pages anymore. Most of the time things are generated on the fly by an application installed into the web server, such as WordPress. So even if you could search the directory, there wouldn't be anything there. instead the file is created only upon request and delivered to the end user, and not written to the directory.
Correct, IF you were sitting on the server and looking at the file system. But that's not how any of these things work so not really a factor.
-
@scottalanmiller said in Security Of Cloud Shared Links:
@Dashrender said in Security Of Cloud Shared Links:
I completely agree with the rest being links.. just the default page loaded when visiting a folder directly be it www.google.com or www.google.com\scottalenmiller
Right, that one is a link because it is listed in DNS. So still a link, just not a generated one
So http://ntg.co/ is still a link, just one linked from DNS.
What? a link from DNS? that's a stretch. So DNS entries are now links?
-
@Dashrender said in Security Of Cloud Shared Links:
@scottalanmiller said in Security Of Cloud Shared Links:
@Dashrender said in Security Of Cloud Shared Links:
I completely agree with the rest being links.. just the default page loaded when visiting a folder directly be it www.google.com or www.google.com\scottalenmiller
Right, that one is a link because it is listed in DNS. So still a link, just not a generated one
So http://ntg.co/ is still a link, just one linked from DNS.
What? a link from DNS? that's a stretch. So DNS entries are now links?
What would you call them? They are a publicly listed link to your site. How do you think of them that would make them something other than a link? If you put a DNS entry into your URL bar, you go to the page, right? What is the A Record list but a collection of links? I mean literally... what else is it?
