Backup and Recovery Goals



  • OK so from this topic it's become abundantly clear that there are way to many pieces at play. With this I need to break out our objectives to this looming project.

    With this assume everything we currently have besides the 1Gb Network backbone is going in the trash. This is a full refresh project.

    Goals

    • First off, I need to virtualize every physical server we have.

    • I need a relatively high availability

    • I also need a very solid backup solution, preferably including fulls and incremental changes.

    • I need hourly incremental backup that are maintained for 3 days.

    • A monthly full to use in the event of an catastrophic event on site.

    • 4 months of network share backups should be kept for recovery.

    • Avoiding taking anything home on fridays would also be ideal

    What we have
    4 physical servers, totaling 5,841 GB of Used Storage. This includes C drives and Network Shares. But totals up to 8TB available.

    Primary File Server

    • 3TB Buffalo Terastation where the bulk of our network shares are mounted, attached as iSCSI to the primary file server

    DC / File Server

    • 1TB Operations Drive on a standalone server
    • 400 GB Share
    • 2TB External USB LaCie
    • a second 2TB External USB LaCie

    Server Software

    • Storage Craft for incremental and fulls.
    • Server OS's 2003 - 2008

    Here are two descriptions that are key to what I'm attempting to achieve.
    The recovery time objective (RTO) is the targeted duration of time and a service level within which a business process must be restored after a disaster (or disruption) in order to avoid unacceptable consequences associated with a break in business continuity.

    • RTO time must <= 1 business day.

    The recovery point objective (RPO) is the age of files that must be recovered from backup storage for normal operations to resume if a computer, system, or network goes down as a result of a hardware, program, or communications failure.

    • RPO must <=1 business hour.


  • @DustinB3403 said:

    RPO must >=1 business hour.

    Greater than 1 hour? 😉



  • @MattSpeller said:

    @DustinB3403 said:

    RPO must >=1 business hour.

    Greater than 1 hour? 😉

    Sign corrected.



  • You are able/willing to lose a month of changes at a time? Is that what I am reading from this? You're RPO and what you mentioned earlier in the post don't match.



  • Are the RTO and RPO goals set by you? Or are they set by management?



  • @coliver said:

    You are able/willing to lose a month of changes at a time? Is that what I am reading from this? You're RPO and what you mentioned earlier in the post don't match.

    No the fulls are only for site catastrophes. Incrementals are to be used to recover files from day to day.

    But I do want a way to backup my VM's primary partitions exclusively for a faster recovery. (if at all possible)



  • @coliver said:

    Are the RTO and RPO goals set by you? Or are they set by management?

    By me and having heard management say we can't live with more. Not my call to argue with them, that's my bosses job.



  • @DustinB3403 said:

    Here are two descriptions that are key to what I'm attempting to achieve.
    The recovery time objective (RTO) is the targeted duration of time and a service level within which a business process must be restored after a disaster (or disruption) in order to avoid unacceptable consequences associated with a break in business continuity.

    • RTO time must <= 1 business day.

    The recovery point objective (RPO) is the age of files that must be recovered from backup storage for normal operations to resume if a computer, system, or network goes down as a result of a hardware, program, or communications failure.

    • RPO must <=1 business hour.

    Something to keep in mind. While every business wants as little downtime as possible and as little data lose as possible, while they may start with that goal, and give the times listed above, once you provide them numbers that provide that level of service, they may come back and change their minds and lower these requirements (i.e. make them longer).



  • @Dashrender said:

    @DustinB3403 said:

    Here are two descriptions that are key to what I'm attempting to achieve.
    The recovery time objective (RTO) is the targeted duration of time and a service level within which a business process must be restored after a disaster (or disruption) in order to avoid unacceptable consequences associated with a break in business continuity.

    • RTO time must <= 1 business day.

    The recovery point objective (RPO) is the age of files that must be recovered from backup storage for normal operations to resume if a computer, system, or network goes down as a result of a hardware, program, or communications failure.

    • RPO must <=1 business hour.

    Something to keep in mind. While every business wants as little downtime as possible and as little data lose as possible, while they may start with that goal, and give the times listed above, once you provide them numbers that provide that level of service, they may come back and change their minds and lower these requirements (i.e. make them longer).

    Right, but that is for management to decide. For right now this is part of the requirements.



  • We currently managed a <=1 Hour RPO with Storage Craft for our network shares.

    I would like to maintain that, and ideally if possible expand it to the entire virtual machines.



  • Excluding replacing hardware. <=1 RPO for VM's and data.

    If hardware dies that's as fast as the truck can get here.



  • I've had to update the original topic, it's not 1 full backup should be kept for recovery. But 4, as this is what we do currently. Well 4 months worth of incremental backups are kept.

    To be clear.



  • Revised again as it's still confusing even when I read it.

    We keep 4 months worth of network share backups for recovery. This data is compressed and readily available to restore from should we need it.

    The 3 day incrementals some how get built into this but these are "always there" to restore from. Nothing special in recovery from this.


  • Service Provider

    @DustinB3403 said:

    • Avoiding taking anything home on fridays would also be ideal

    IronMountain for the win.


  • Service Provider

    @DustinB3403 said:

    Primary File Server

    • 3TB Buffalo Terastation where the bulk of our network shares are mounted, attached as iSCSI to the primary file server

    That would be your SAN. iSCSI can't be a file server. It's just storage behind the actual file server which must be a VM somewhere.



  • @scottalanmiller said:

    That would be your SAN. iSCSI can't be a file server. It's just storage behind the actual file server which must be a VM somewhere.

    In his case it's a 2008 Server that we know of from the other thread.



  • Are you including the 3 TB of data on the Terrastation in your 6 TB of data?



  • The primary file server is directly connected to the Buffalo drive. The server simply shares out the attached stored.

    The primary DC has multiple functions on it, Spiceworks, AD, and 2 network shares.

    The 3rd server is acting as a HyperV host for 0365 / AD integration and a Reporting server.

    @Dashrender said:

    Are you including the 3 TB of data on the Terrastation in your 6 TB of data?

    Yes.



  • In the other thread you mentioned that StorageCraft was installed on it's own box.. did I misunderstand that?



  • So 3 TB of your data is coming from the network shares - where is the rest of it? More shares? on other servers? DBs?


  • Service Provider

    @DustinB3403 said:

    @coliver said:

    Are the RTO and RPO goals set by you? Or are they set by management?

    By me and having heard management say we can't live with more. Not my call to argue with them, that's my bosses job.

    A financial person working for the CFO's office should be making sure RTO and RPO are never said on the planning stages (see the other thread before this one from @Dashrender where I mentioned why RTO and RPO can't be used in planning because it leas to disaster.) The financial people should understand intrinsically that reducing risk and the cost to do so are a sliding scale and it is always an assessment based on a combination of the two and any statement of RTO or RPO desires is just crazy. Hopefully even junior financial people are available to assist you if the CFO is too busy to make sure that managers never make this kind of mistake.


  • Service Provider

    @DustinB3403 said:

    The primary file server is directly connected to the Buffalo drive. The server simply shares out the attached stored.

    Using a slow SAN behind a file server makes backups that much harder before you have more bottlenecks to address. No good way to make up a SAN directly, so the back AND the restore has to go through the fileserver.



  • @scottalanmiller said:

    @DustinB3403 said:

    @coliver said:

    Are the RTO and RPO goals set by you? Or are they set by management?

    By me and having heard management say we can't live with more. Not my call to argue with them, that's my bosses job.

    A financial person working for the CFO's office should be making sure RTO and RPO are never said on the planning stages (see the other thread before this one from @Dashrender where I mentioned why RTO and RPO can't be used in planning because it leas to disaster.) The financial people should understand intrinsically that reducing risk and the cost to do so are a sliding scale and it is always an assessment based on a combination of the two and any statement of RTO or RPO desires is just crazy. Hopefully even junior financial people are available to assist you if the CFO is too busy to make sure that managers never make this kind of mistake.

    How do you pick a starting point without setting an objective, even if it is just a starting point.

    What other objectives would you prefer to use to base a design on?


  • Service Provider

    @Dashrender said:

    How do you pick a starting point without setting an objective, even if it is just a starting point.

    This is a mental exercise. I will guess that you are a perceiver, like me. I cannot handle an ambiguous starting point and need someone else, a planner, to set an objective and then I can say "that makes no sense, let's adjust to this..." but I cannot provide the starting point on my own.

    This is a mental deficiency that we share. What we need is someone who is a planner. Someone who can set that goal for us to tweak.


  • Service Provider

    @Dashrender said:

    What other objectives would you prefer to use to base a design on?

    You don't, you build a curve and adjust the spending up or down until you find the mix of risk and of cost that is best for your specific business.


  • Service Provider

    This is purely a financial exercise and not an IT one. IT people are not trained to understand this or make this decision. This is for the CFO and his team. The IT perspective here is to provide info to create the graph.



  • @scottalanmiller said:

    @Dashrender said:

    How do you pick a starting point without setting an objective, even if it is just a starting point.

    This is a mental exercise. I will guess that you are a perceiver, like me. I cannot handle an ambiguous starting point and need someone else, a planner, to set an objective and then I can say "that makes no sense, let's adjust to this..." but I cannot provide the starting point on my own.

    This is a mental deficiency that we share. What we need is someone who is a planner. Someone who can set that goal for us to tweak.

    Yes, we share that in common.


  • Service Provider

    @Dashrender said:

    Yes, we share that in common.

    Lots of IT people are. Perceiving is an important skill for doing good triage. Just one of those life tricks, learning to find a planner to be your compliment.

    I'm lucky that I have @art_of_shred at work and @dominica at home who are both strong planners to do that side of things and they have me for perception roles.



  • @scottalanmiller said:

    This is purely a financial exercise and not an IT one. IT people are not trained to understand this or make this decision. This is for the CFO and his team. The IT perspective here is to provide info to create the graph.

    OK, this is true to a point, maybe a greater one that I'd like to admit.

    But referencing Darren's CEO/CFO talk at SpiceWorld, IT presents options to management, sometimes for homegrown projects. For management provided projects it's pretty easy, as they should be providing the starting point to help IT get the graph started. An IT homegrown project requires the start to come from within, and probably really also requires them to find the financial side as well.


  • Service Provider

    @Dashrender said:

    @scottalanmiller said:

    This is purely a financial exercise and not an IT one. IT people are not trained to understand this or make this decision. This is for the CFO and his team. The IT perspective here is to provide info to create the graph.

    OK, this is true to a point, maybe a greater one that I'd like to admit.

    But referencing Darren's CEO/CFO talk at SpiceWorld, IT presents options to management, sometimes for homegrown projects. For management provided projects it's pretty easy, as they should be providing the starting point to help IT get the graph started. An IT homegrown project requires the start to come from within, and probably really also requires them to find the financial side as well.

    I don't think that it should, even for IT-initiated projects (we can discuss why IT should probably not initiate many projects in general elsewhere.) IT just lacks the info and power here. The management or financial teams should have some resource somewhere to help with things. This is difficult to explain in a purely abstract way. Can you think of an example where you feel that IT would need to provide a starting point?