Cloud Hosted Storage



  • So following along with my topic from the other day I was asked how is the data accessed.

    Without having used a service such as BackBlaze or Amazon S3 cloud services I'm just guessing. It was asked if it's possible to mount a service to a server as a local share.

    I can only infer that "mounting" a service like this impossible in a traditional Microsoft File Share sense.

    In doing some research it appears that you have to run a VM on Amazon's service to act as a file server. You can configure SFTP access so tools like WinSCP could be used to access the data. The information I've found is that generally this is a cost per hour. But as you'll see below there are listed AWS Free Tier Eligible Software which cost $0.00/Hour. . .

    There are a list of different options here of AWS Free Tier Eligible Software. What I assume this means is that I could run an SFTP server on Amazon S3 free of charge if using CentOS for as long as we have the service. Is this correct?

    As a general inquiry are all online cloud services like this when looking at these sorts of options?



  • You could look at http://www.cloudberrylab.com/ I think they do local mounting.

    In grad school I wrote a daemon that monitored a folder, in a Linux file system, and upload new files to a S3 instance. It would run when a file was written to that folder. It worked maybe 80% of the time.



  • @coliver So we'd use CloudBerry's application on our systems, and still run Amazon S3 for the storage, correct?



  • @DustinB3403 said:

    @coliver So we'd use CloudBerry's application on our systems, and still run Amazon S3 for the storage, correct?

    Last time I looked at cloudberry it mounted as a drive letter to a system and then uploaded files that went into it to Amazon S3.



  • Although it doesn't look like it does this anymore? I can't seem to find that functionality on their website now.



  • Just reading the documentation on it, it appears that it acts much like WinSCP (for the paid version).

    I haven't found anything about it being able to mount as a share. Will keep digging into it.

    Do you not use cloudberry / Amazon S3?



  • @DustinB3403 said:

    Just reading the documentation on it, it appears that it acts much like WinSCP (for the paid version).

    I haven't found anything about it being able to mount as a share. Will keep digging into it.

    Do you not use cloudberry / Amazon S3?

    Nope, I trialed it a several years ago when it was much simpler. It worked similar to what I described.



  • http://www.cloudberrylab.com/amazon-cloud-server-drive.aspx

    This is what they originally released.



  • @coliver said:

    http://www.cloudberrylab.com/amazon-cloud-server-drive.aspx

    This is what they originally released.

    See that seems, almost too good for what it is. $60 bucks one time, to mount a remote share on our server. So our users can access it. Do you know how ACL works with that?

    How are domain users granted access to the share / S3 service. Since S3 has it's own user permissions. (I'm guessing wide open for everyone) if we went this route.

    To boot, I'm thinking about this and trying to come up with a plan for archiving old data to a cloud storage provider, I don't know if it's wise to grant "everyone" access to it.

    For multiple reasons data explosion (people just abusing the storage and putting everything up there), accidental deletion, Hands-with-a-Spoon syndrome being the top three concerns.



  • @DustinB3403 said:

    @coliver said:

    http://www.cloudberrylab.com/amazon-cloud-server-drive.aspx

    This is what they originally released.

    See that seems, almost too good for what it is. $60 bucks one time, to mount a remote share on our server. So our users can access it. Do you know how ACL works with that?

    How are domain users granted access to the share / S3 service. Since S3 has it's own user permissions. (I'm guessing wide open for everyone) if we went this route.

    To boot, I'm thinking about this and trying to come up with a plan for archiving old data to a cloud storage provider, I don't know if it's wise to grant "everyone" access to it.

    For multiple reasons data explosion (people just abusing the storage and putting everything up there), accidental deletion, Hands-with-a-Spoon syndrome being the top three concerns.

    I'm not sure about any of that. Sorry. They do have a free trial not sure if that is worth it or not?



  • @DustinB3403 said:

    Without having used a service such as BackBlaze or Amazon S3 cloud services I'm just guessing. It was asked if it's possible to mount a service to a server as a local share.

    Not directly, no. This is not a file server in the traditional sense. This is not how this kind of storage should be thought of.



  • @DustinB3403 said:

    In doing some research it appears that you have to run a VM on Amazon's service to act as a file server.

    No, that is an option but nothing makes that "have to be" the way that you do it. Even an Amazon VM needs a way to consume that data and to present that data. You can do this in other ways.

    Bigger question... why do you want S3 presented as an SMB share?



  • @DustinB3403 said:

    @coliver So we'd use CloudBerry's application on our systems, and still run Amazon S3 for the storage, correct?

    Yes. CloudBerry is just software that talks to S3 and makes it look like a local drive.



  • @DustinB3403 said:

    See that seems, almost too good for what it is. $60 bucks one time, to mount a remote share on our server.

    Most people think that that cost is crazy, you think it sounds too good to be true? CloudBerry is often criticized for being ridiculously expensive for how little it does.



  • @DustinB3403 said:

    How are domain users granted access to the share / S3 service. Since S3 has it's own user permissions. (I'm guessing wide open for everyone) if we went this route.

    You don't normally. This is a cloud storage service, you don't just open it up to end users.

    If you want to do that you need something far more complex. CloudBerry makes their NAS-like service, AD Bridge, specifically for this. This makes S3 look to your users like a NAS device.



  • @DustinB3403 said:

    To boot, I'm thinking about this and trying to come up with a plan for archiving old data to a cloud storage provider, I don't know if it's wise to grant "everyone" access to it.

    That's why S3 is complicated, it is a raw tool for IT to build tools on top of, not something to expose to end users 99% of the time.



  • I think a bigger part of your problem here is that you have not actually decided what you are trying to accomplish. Because of that, you keep mixing up use cases and scenarios.



  • Well I would like to use the storage for just Archival. My boss asked the above question as to whether its mountable as a share.

    I think mounting it as a share is a bad idea, and would prefer to manage the storage from within IT. Were we upload/download files for the company.

    Removing everyone else from the mix.



  • I'm having a hard time understanding how to use storage that isn't treated like an SMB/NFS share.

    What you just put it somewhere in the cloud for safe keeping? You use SFC if for some reason you need to bring copies back to your local system?



  • For simplicity cloud storage can run any file system you need. At least from what I'm seeing with Amazon S3.

    So you could use something like WinSCP to connect to Amazon S3 service of yours assuming its configured as NFS / SMB Share and copy down or upload your file(s).

    It's pretty straight forward from what I can determine, but I'm looking for is just a simple, safe online cloud storage platform that manages all of the back-end 'stuff'. I don't care about the possibilities, so long as the data is safely stored remotely, that is easily recovered from, and that can scale.



  • @DustinB3403 said:

    Well I would like to use the storage for just Archival. My boss asked the above question as to whether its mountable as a share.

    I think mounting it as a share is a bad idea, and would prefer to manage the storage from within IT. Were we upload/download files for the company.

    Removing everyone else from the mix.

    Do that, and then you can spin up the Amazon SFTP instance and even if it is not free, you can shut just turn it on, perform the upload of new data, and then turn it back off. That is the whole point of AWS billing. Pay only when it is used.



  • @Dashrender said:

    I'm having a hard time understanding how to use storage that isn't treated like an SMB/NFS share.

    Think RESTful API style storage.



  • @scottalanmiller said:

    @Dashrender said:

    I'm having a hard time understanding how to use storage that isn't treated like an SMB/NFS share.

    Think RESTful API style storage.

    Try again, this time with a link explaining RESTful API 🙂




  • Vendor

    @coliver said:

    ** You could look at http://www.cloudberrylab.com/ I think they do local mounting.**

    In grad school I wrote a daemon that monitored a folder, in a Linux file system, and upload new files to a S3 instance. It would run when a file was written to that folder. It worked maybe 80% of the time.

    We used this one before but found unreliable. Something goes wrong with Internet connection and it's very difficult to recover. WebDAV, SMB3 or application-specific integration (like one Veeam had done to their cloud connectivity also using ref'd software before) is a way to go. At least you can control data on your side @ all stages 🙂



  • @KOOLER said:

    @coliver said:

    ** You could look at http://www.cloudberrylab.com/ I think they do local mounting.**

    In grad school I wrote a daemon that monitored a folder, in a Linux file system, and upload new files to a S3 instance. It would run when a file was written to that folder. It worked maybe 80% of the time.

    We used this one before but found unreliable. Something goes wrong with Internet connection and it's very difficult to recover. WebDAV, SMB3 or application-specific integration (like one Veeam had done to their cloud connectivity also using ref'd software before) is a way to go. At least you can control data on your side @ all stages 🙂

    I was thinking that WebDav would be the perfect style storage for this. Windows can generally mount WebDav as a drive as well. Although you would probably need to setup a small Linux web server with something like SabreDav running.



  • WebDAV is supposed to work like that but it depends on the Windows client versions that you are using as to whether or not it will work reliably mounting it as a standard mapped drive.



  • @DustinB3403 said:

    See that seems, almost too good for what it is. $60 bucks one time, to mount a remote share on our server.

    I just realized that maybe you were thinking that this was a one time fee. It's a one time fee for the software but you still pay for Amazon S3 the same as you always would. So it is all of the costs of S3 plus $60, one time, per machine that will access it.



  • @Dashrender said:

    I'm having a hard time understanding how to use storage that isn't treated like an SMB/NFS share.

    Think about OneDrive, Google Drive, DropBox, etc. None of those use SMB or NFS shares. All require extra software running on your computer to interface between your desktop and the hosted storage.

    In all of those cases, they are using HTTPS or something similar and just making PUTS and GETS on the files. Then they have local software that presents that to you as if it was a local drive.



  • @DustinB3403 said:

    For simplicity cloud storage can run any file system you need. At least from what I'm seeing with Amazon S3.

    No, not at all. There are no filesystems at all on "cloud storage." That's a terrible term as it is unrelated to cloud in any way. It's object storage that we are discussing.

    Amazon S3, Amazon Glacier, BackBlaze, Azure Storage, etc. is all the same, it is object storage. Conceptually there is no such thing as a file system on them. You can't even think of the storage that way, let alone attempt to apply a filesystem.

    You are picturing a SAN delivering block storage. That's not what this is. And what everyone is hoping for is a network file system (NFS, SMB, AFP, etc.) which this is not. Object storage is a third storage type and quite different.


Log in to reply