@Jimmy9008 said in Marketing - Video Editing Storage:
Hi all,
I am being asked to find a storage solution for our video editing department. They use various Adobe tools on Mac clients.
They have around 40TB of files they use, with the largest files being RAW files of 300 GB - 400 GB. Average file sizes are 10 GB - 20 GB.
1/2 of this is archive and can be stored on cloud storage for pull down should they require again. Leaving around 20TB live data and 20TB in archive.
Originally, I was looking at proposing a 20 - 30 TB NAS populated with SSDs in the local office, with 10 Gbps NIC. This would provide high speed local access over the LAN to 6 marketing users.
Marketing said speed was the main concern. Accessing the large files is currently slow causing many delays. They have so far been sharing USB 3.0 devices between each other, without backups.
Our CIO is now pushing for cloud only solutions for the storage where marketing can check in/out the files they need, killing my NAS idea. I have concerns on this but am open, so would like some advice. What have you used or what would you suggest to use to provide this?
I am concerned that when an editor wants to do something with the 300 GB file, that will have to pull down from the cloud, bringing our WAN link to a crawl until the download finishes. Then, once they have finished editing, they have to upload the end product, again causing bandwidth issues. This is exacerbated shoudl multiple editors be pulling multiple files at the same time. Even at 500 Mbps with no overhead that is around 1.5h of time to sit and wait for a 300 GB file whilst everybody else is affected on the same WAN link.
I assume the'video editing' cloud storage providers are like OneDrive or DropBox, where each machine has a local cache so do not have to pull from cloud each time. But, wouldnt that mean that our video editing workstations all need to have a 20 TB local drive to store this local cache?
Kind of all over the place on this one, any ideas folks?
You need to present the math for the CIO.
It's unrealistic to assume you can use your entire bandwith for one download. It's also completely unrealistic to assume you can get 500 Mbps sustained from whatever cloud storage you have.
I would say you're lucky to have 100 Mbit/s sustained. So 10MB/s of data and a little over 8 hours for a 300GB file. 50 Mbit/s is however a more realistic number.
Next up is the maximum file size limits on cloud storage. Microsoft for example is 250GB maximum on Onedrive and Sharepoint. In other words - you can't store your video files there.
Using cloud for archival storage you may never need is fine. But for working files that the editing team is sharing, you must have those files on the LAN.
You should run a test. Sign up for a cloud storage solution and take one of those 400GB RAW files and upload it. And then download it. See what happens and how long it takes.