Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?
-
Hello all,
Unlike for any local or onsite backup with NAS, cloud backup seems to be bit difficult to understand, estimate, propose to management and use in real-time.
Here my concern is about how suitable Google Cloud Storage for us to maintain Offsite Backup in the view Bandwidth or performance. Second issue is about Pricing, in general, finance guys expect how exactly it will cost/per year, and they don't want to understand Pay As you Go ! (even me maybe
?)Following is the log of Backup I have done from QNAP NAS to Google Cloud Storage as a test :
You can see, it used very less bandwidth, and took around 3 days to copy 120GB.
I am not seeing good options to control bandwidth things in Google Cloud app for QNAP, but I can tune (to allocate good bandwidth) to QNAP with Firewall.
So what are you thoughts, how do you check for your suitability ?
Many Thanks !
-
Where was the bottleneck on that transfer speed? Was it the QNAP? Google themselves (doubtful)? your ISP?
-
What is your daily rate of change?
How often will you push to the cloud for backup?What is Google's pricing?
-
@Dashrender said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
Where was the bottleneck on that transfer speed? Was it the QNAP? Google themselves (doubtful)? your ISP?
About bandwidth, we have 40 Mbps down and 20 Mbps Up. I have started pushing the data to Google Cloud from QNAP at working hours, that may have impact, but still 3 days ? (It got around 3 nights if working hours is an issue).
I am not sure what is the cause. One option in my hand is allocating dedicated bandwidth to QNAP NAS on Firewall.
How do you verify at QNAP and Google ?
-
I've never used a QNAP - can you connect to it and tell it to send say 1 GB to a local onsite server and see what your bandwidth usage is?
-
@Dashrender said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
What is your daily rate of change?
How often will you push to the cloud for backup?What is Google's pricing?
We are running incremental backups to Onsite (to QNAP NAS), based on that I am seeing on average of 4GB it's showing as data size of incremental backup.
How often, I am thinking of pushing to Cloud from NAS weekly once or twice in a month, whichever is feasible.
Pricing :
We are keeping 30 days data backup. So backup software may keep up to 60 days backup and rotate. So as of today, the backup data size is 8.5TB, so 59th day backup it will come down to around 5TB. So in process of rotation, the data size will increase and decrease. Of course, the growth of actual data is there, and approximately 4GB.
I have no idea of how to use Google Calculator and get pricing, so I can submit the proposal to manager.
-
How much actual data do you have? 5 TB?
It seems odd that your backups would grow to 8.5 TB then drop to 5 TB, I assume this means your backups can't do daily pruning of your retention policy?
So you have 4 GB of changes a day - not a small amount, but not huge either.
If you're going to sync just once a week, what's your weekly change rate? that's a number you're going to need to give Google to figure out pricing, unless the QNAP is backing up all the incrementals from the week on that single weekly sync.
-
@Dashrender said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
How much actual data do you have? 5 TB?
It seems odd that your backups would grow to 8.5 TB then drop to 5 TB, I assume this means your backups can't do daily pruning of your retention policy?
How is this odd? Full backup = 5TB. Then add incremenatls, bring it up to 8.5TB.
Then a new full backup runs and then purges the old full and incrementals bringing it back to 5TB.
-
@JaredBusch said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
@Dashrender said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
How much actual data do you have? 5 TB?
It seems odd that your backups would grow to 8.5 TB then drop to 5 TB, I assume this means your backups can't do daily pruning of your retention policy?
How is this odd? Full backup = 5TB. Then add incremenatls, bring it up to 8.5TB.
Then a new full backup runs and then purges the old full and incrementals bringing it back to 5TB.
I did specifically say that I was assuming he wasn't using daily pruning - the lack of that is what "I" found odd. Nothing more was said or implied.
-
@Dashrender said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
How much actual data do you have? 5 TB?
It seems odd that your backups would grow to 8.5 TB then drop to 5 TB, I assume this means your backups can't do daily pruning of your retention policy?
So you have 4 GB of changes a day - not a small amount, but not huge either.
If you're going to sync just once a week, what's your weekly change rate? that's a number you're going to need to give Google to figure out pricing, unless the QNAP is backing up all the incrementals from the week on that single weekly sync.
@JaredBusch said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
@Dashrender said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
How much actual data do you have? 5 TB?
It seems odd that your backups would grow to 8.5 TB then drop to 5 TB, I assume this means your backups can't do daily pruning of your retention policy?
How is this odd? Full backup = 5TB. Then add incremenatls, bring it up to 8.5TB.
Then a new full backup runs and then purges the old full and incrementals bringing it back to 5TB.
@Dashrender , @JaredBusch is right. The Actual data is around 5TB and incremental backup takes up to 60 days and purge 30 days backup and keeps 30 days anytime.
-
So backup data is going up to 8.5 TB now, and of course this 8.5 TB is not stable and will change as actual data changes.
So should I consider 8.5 TB ? or 10TB (for safer side) as my Cloud Storage need ?
-
@openit said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
So backup data is going up to 8.5 TB now, and of course this 8.5 TB is not stable and will change as actual data changes.
So should I consider 8.5 TB ? or 10TB (for safer side) as my Cloud Storage need ?
I'm pretty sure Google changes for what you actually use. But for presentation purposes, it might be best to just use 10 TB so you don't sell it short on possible cost.
-
@openit said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
Unlike for any local or onsite backup with NAS, cloud backup seems to be bit difficult to understand, estimate, propose to management and use in real-time.
That's because "cloud backup" isn't a thing. It's an amorphous concept, not even a single concept let alone a single thing.
-
@openit said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
I am not sure what is the cause. One option in my hand is allocating dedicated bandwidth to QNAP NAS on Firewall.
Never dedicate bandwidth, use QoS instead. Dedicated just cripples everything else.
-
@scottalanmiller said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
@openit said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
I am not sure what is the cause. One option in my hand is allocating dedicated bandwidth to QNAP NAS on Firewall.
Never dedicate bandwidth, use QoS instead. Dedicated just cripples everything else.
Sorry, I am no expert in Networks, actually I meant for QoS by saying Dedicated. I understand it could be different in technical meaning like, QoS will use bandwidth only when it needs something like that ?
-
@openit said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
@scottalanmiller said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
@openit said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
I am not sure what is the cause. One option in my hand is allocating dedicated bandwidth to QNAP NAS on Firewall.
Never dedicate bandwidth, use QoS instead. Dedicated just cripples everything else.
Sorry, I am no expert in Networks, actually I meant for QoS by saying Dedicated. I understand it could be different in technical meaning like, QoS will use bandwidth only when it needs something like that ?
Correct. A lot of people do reserved bandwidth instead of QoS and really mess up their networks, it's a common thing to get wrong so very important to use the right terms - especially since a lot of routers offer both.
-
So I am trying to estimate the Google Cloud Storage pricing with their calculator.
Requirements :
Region : (looking for cheapest one) - selected South Carolina as a test. which one is cheapest location ?
Storage Capacity : 10TB. And space could be increased without any interruption, right ?
Redundancy : I believe this multi-regional is for Redundancy, which means our data will be maintained in two or more locations. I am not going with Redundancy option.
Type of Storage : Because I don't want cold storage, I have chosen "Cloud Storage Nearline"
That's it ?
-
@scottalanmiller @Dashrender @JaredBusch @DustinB3403
I have just filled Location and Cloud Storage nearline, no idea about other options like operations etc.
-
How about restore options below storage ?
Should I include this estimate too ? For retrieving of data, we will require to retrieve only at the time of unsuccessful restoration from Onsite backup. And sometimes I want to retrieve to test.
One more thing I believe to be considered is, the backup we are going to take is Backup Files (Incremental Backup), so all files needs to downloaded at the time retrieval, so downloading 10TB will be horrible and may take ages
?
-
@scottalanmiller said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
@openit said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
@scottalanmiller said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
@openit said in Google Cloud as offsite backup. How to check if it's suitable for us or no and pricing ?:
I am not sure what is the cause. One option in my hand is allocating dedicated bandwidth to QNAP NAS on Firewall.
Never dedicate bandwidth, use QoS instead. Dedicated just cripples everything else.
Sorry, I am no expert in Networks, actually I meant for QoS by saying Dedicated. I understand it could be different in technical meaning like, QoS will use bandwidth only when it needs something like that ?
Correct. A lot of people do reserved bandwidth instead of QoS and really mess up their networks, it's a common thing to get wrong so very important to use the right terms - especially since a lot of routers offer both.
Yup, same concept with phone systems. QoS is the best way to make sure they work even if everything else is congested.
