Historical Data Retention
-
We are running into so data retention questions with historical data and I was curious if anyone had any experience and insight into industry standards or best practices when it comes to historical data retention and databases. What I'm wondering is how long the big boys keep their data live, how they offload it and what they do with old data. For example does Amazon have all their historical purchase data since they started or do they rotate it out after a certain amount of time? Anyway, it just got me thinking so any insight or ideas are welcomed. Thanks.
-
Generally only as long as required by law, and not a day longer.
For financial firms it is something liek 3 years on site retention, 7 years offsite retention. -
@momurda said in Historical Data Retention:
Generally only as long as required by law, and not a day longer.
For financial firms it is something liek 3 years on site retention, 7 years offsite retention.If you happen to work where retention isnt required by law, you probably arent going to do it.
-
@momurda said in Historical Data Retention:
@momurda said in Historical Data Retention:
Generally only as long as required by law, and not a day longer.For financial firms it is something liek 3 years on site retention, 7 years offsite retention.
If you happen to work where retention isnt required by law, you probably arent going to do it.
Well you shouldn't at least, because if you have it, it can be subject to discovery
ifwhen you get sued for something. -
@momurda said in Historical Data Retention:
Generally only as long as required by law, and not a day longer.
This exactly. Old data is bad in most cases.
-
@JaredBusch said in Historical Data Retention:
@momurda said in Historical Data Retention:
@momurda said in Historical Data Retention:
Generally only as long as required by law, and not a day longer.For financial firms it is something liek 3 years on site retention, 7 years offsite retention.
If you happen to work where retention isnt required by law, you probably arent going to do it.
Well you shouldn't at least, because if you have it, it can be subject to discovery
ifwhen you get sued for something.Yup. Run away from unnecessarily stored data.
-
@momurda said in Historical Data Retention:
For financial firms it is something liek 3 years on site retention, 7 years offsite retention.
3 years for standard data. 7 years for actual trades or whatever. No onsite/offsite stuff. Not even a concept of that. What's onsite to an institution when the data is never onsite in the first place?
-
On the other hand, old data can be very useful for architect/engineering type companies. They may not store the project records forever, but the drawings have value. In that case the space on a server to store an old drawing compared to a current one is laughable since the file sizes keep increasing.
-
Correct. We manufacture products that typically last 30 years or so. We occasionally get requests for spare parts for products that we sold 50 years ago.
It's a pain. Last week a user said "I'm having trouble reviewing a quote I sent someone a while back." Reason: the quote was created in 2002 using Microsoft Office Binder (OBD). I didn't even know what an .OBD file was.
-
We are required to keep engineering docs for 60 years since our product is guaranteed for that long. Until a few years ago they still used microfiche, but we are using a certain type of DVD now that supposedly lasts for like hundreds of years.
-
I believe best practice is to keep data retention long enough for legal needs, and long enough for business needs... such as the examples @Carnival-Boy and @stacksofplates gave. Each individual business should decide what data could be needed in the future and plan a retention plan appropriately.
I don't think there's a standard amount of time as a best practice, as it varies way too much business to business, state to state, country to country, due to so many reasons.
Every business will be different.
-
Thanks for all the info. I appreciate it. We have been a small/young division since we started but now we are getting larger and older meaning questions like "Do we need or will we ever need all this data for customers that are no longer using our services?" are needing to be asked. So thanks again, all your replies have been very helpful.
-
Most important thing is a written policy that gets followed.
-
@scottalanmiller said in Historical Data Retention:
Most important thing is a written policy that gets followed.
Can't stress this enough!
-
@Carnival-Boy said in Historical Data Retention:
Correct. We manufacture products that typically last 30 years or so. We occasionally get requests for spare parts for products that we sold 50 years ago.
It's a pain. Last week a user said "I'm having trouble reviewing a quote I sent someone a while back." Reason: the quote was created in 2002 using Microsoft Office Binder (OBD). I didn't even know what an .OBD file was.
That's a new format for me, too.
-
@Mike-Davis Yes!, this happen all the time with all the construction companies I work with. With some staff using Project 97 since it still works on Windows 7.... I was floored
-
@dbeato said in Historical Data Retention:
@Mike-Davis Yes!, this happen all the time with all the construction companies I work with. With some staff using Project 97 since it still works on Windows 7.... I was floored
Discovering new "old" formats that you've never seen before?
-
@scottalanmiller not really, just some I don't remember
-
@dbeato said in Historical Data Retention:
@scottalanmiller not really, just some I don't remember
I have started to forget about MS Project completely. It's been so long since I knew of anyone actively considered using it. So many alternatives that are not so costly and don't have old fashioned web clients have made it fall into the background of technology history. Once people mention it, I remember it. But ask me about project management software and I never think of it anymore.
-
@scottalanmiller said in Historical Data Retention:
@dbeato said in Historical Data Retention:
@scottalanmiller not really, just some I don't remember
I have started to forget about MS Project completely. It's been so long since I knew of anyone actively considered using it. So many alternatives that are not so costly and don't have old fashioned web clients have made it fall into the background of technology history. Once people mention it, I remember it. But ask me about project management software and I never think of it anymore.
Which ones are on your table now?
