If It Ain't Broke, Don't Fix It
-
A good way to have continuous improvement is to follow an industry standard such as NIST, CIS, etc. If senior management supports getting to and maintaining a standard this wouldn't be an issue.
-
In terms of IT systems, broken means unnecessary exposed to hacking, data theft, data loss, downtime and inefficiencies. In the real world, we should be considering the system to be broken the moment that maintenance is needed. How much ransomware would not be a threat today if systems were simply properly maintained? As IT we need to stand up and explain that unmaintained systems are already broken, disaster just hasn’t struck yet.
This is probably the most important paragraph in that entire piece!
This is something every person should be taught about anything, everything they deal with. -
If we were to follow the mantra of “if it ain’t broke, don’t fix it” in IT, we wait until our data was stolen to patch vulnerabilities, or wait until data was unrecoverable to see if we had working backups. Of course, that makes no sense. But this is what is often suggested when people tell you not to fix your systems until they break – they are telling you to let them break! Push back, don’t accept that kind of advice. Explain that the purpose of good IT maintenance is to avoid systems breaking whenever possible. Avoiding disaster, rather than inviting it.
I think this could use a dose of change from the previous paragraph.
Instead of
Explain that the purpose of good IT maintenance is to avoid systems breaking whenever possible.
Explain that the purpose of good IT maintenance is to avoid bad things happening because of discovered broken things - or something like that. It needs to be reinforced that once a vulnerability is discovered, you are broken, so the it ain't broke part fails.
-
@dashrender said in If It Ain't Broke, Don't Fix It:
In terms of IT systems, broken means unnecessary exposed to hacking, data theft, data loss, downtime and inefficiencies. In the real world, we should be considering the system to be broken the moment that maintenance is needed. How much ransomware would not be a threat today if systems were simply properly maintained? As IT we need to stand up and explain that unmaintained systems are already broken, disaster just hasn’t struck yet.
This is probably the most important paragraph in that entire piece!
This is something every person should be taught about anything, everything they deal with.Agreed.
One of the things we learn in IT Risk management is that your asset is valued by how much business you will lose in the event of a failure.
Alot of people value their systems by their initial cost or cost of replacement. With this mentality you will always try to milk the longevity of the product to trick yourself into thinking you have more cost benefit then really exists, and your potential for risk just continues to grow.
If looking at actual business loss, the no brainer decision will always be proper maintenance and security.
-
@DustinB3403 said in If It Ain't Broke, Don't Fix It:
I haven't read the article yet, but I am already assuming your stance. That the statement comes from a stance of fear, uncertainly about updating and patching (and rebooting).
More a misunderstanding of maintenance, but that too.
-
@dafyre said in If It Ain't Broke, Don't Fix It:
"Must like a car, but dramatically moreso. "
Should that be "Much like a car ?"
===
I think I agree with the premise of your article. One of the reasons I have always used that phrase when dealing with pretty much anything technology is along the lines of let's not change stuff just because we can.While people often say that that is why they use it, I've never seen it used in a situation where that would apply. They use it to avoid updating software or hardware that is aging. I've never seen it about "changing" but always about "updating."
-
@DustinB3403 said in If It Ain't Broke, Don't Fix It:
@dafyre said in If It Ain't Broke, Don't Fix It:
"Must like a car, but dramatically moreso. "
Should that be "Much like a car ?"
===
I think I agree with the premise of your article. One of the reasons I have always used that phrase when dealing with pretty much anything technology is along the lines of let's not change stuff just because we can.Let's have a reason for changing stuff, and let's make it better than it was before. Because then, we're not fixing something that is broken -- we are improving (or securing) something that needs to be improved (secured).
But in any case, using the statement "If it ain't broke, don't fix" could easily be construed as "well our server hasn't been patched, but is still working. Why bother patching it?"
The statement leaves to much open space for interpretation.
That's how I see it used all of the time. Moreso about bigger updates than patches, but the same basic thing. Basically... we've not yet lost our data and had downtime, so let's not do anything to prevent it until something truly awful happens.
-
@IRJ said in If It Ain't Broke, Don't Fix It:
@dashrender said in If It Ain't Broke, Don't Fix It:
In terms of IT systems, broken means unnecessary exposed to hacking, data theft, data loss, downtime and inefficiencies. In the real world, we should be considering the system to be broken the moment that maintenance is needed. How much ransomware would not be a threat today if systems were simply properly maintained? As IT we need to stand up and explain that unmaintained systems are already broken, disaster just hasn’t struck yet.
This is probably the most important paragraph in that entire piece!
This is something every person should be taught about anything, everything they deal with.Agreed.
One of the things we learn in IT Risk management is that your asset is valued by how much business you will lose in the event of a failure.
Alot of people value their systems by their initial cost or cost of replacement. With this mentality you will always try to milk the longevity of the product to trick yourself into thinking you have more cost benefit then really exists, and your potential for risk just continues to grow.
If looking at actual business loss, the no brainer decision will always be proper maintenance and security.
This is related to sunk cost. They perceive the sunk cost but ignore the business costs, which are the ones that matter.
-
@scottalanmiller said in If It Ain't Broke, Don't Fix It:
@DustinB3403 said in If It Ain't Broke, Don't Fix It:
@dafyre said in If It Ain't Broke, Don't Fix It:
"Must like a car, but dramatically moreso. "
Should that be "Much like a car ?"
===
I think I agree with the premise of your article. One of the reasons I have always used that phrase when dealing with pretty much anything technology is along the lines of let's not change stuff just because we can.Let's have a reason for changing stuff, and let's make it better than it was before. Because then, we're not fixing something that is broken -- we are improving (or securing) something that needs to be improved (secured).
But in any case, using the statement "If it ain't broke, don't fix" could easily be construed as "well our server hasn't been patched, but is still working. Why bother patching it?"
The statement leaves to much open space for interpretation.
That's how I see it used all of the time. Moreso about bigger updates than patches, but the same basic thing. Basically... we've not yet lost our data and had downtime, so let's not do anything to prevent it until something truly awful happens.
But that is why we have snapshots and backups and other ways of recovering from the "truly awful" that it is absurd to not do updates.
-
@dafyre said in If It Ain't Broke, Don't Fix It:
@scottalanmiller said in If It Ain't Broke, Don't Fix It:
@DustinB3403 said in If It Ain't Broke, Don't Fix It:
@dafyre said in If It Ain't Broke, Don't Fix It:
"Must like a car, but dramatically moreso. "
Should that be "Much like a car ?"
===
I think I agree with the premise of your article. One of the reasons I have always used that phrase when dealing with pretty much anything technology is along the lines of let's not change stuff just because we can.Let's have a reason for changing stuff, and let's make it better than it was before. Because then, we're not fixing something that is broken -- we are improving (or securing) something that needs to be improved (secured).
But in any case, using the statement "If it ain't broke, don't fix" could easily be construed as "well our server hasn't been patched, but is still working. Why bother patching it?"
The statement leaves to much open space for interpretation.
That's how I see it used all of the time. Moreso about bigger updates than patches, but the same basic thing. Basically... we've not yet lost our data and had downtime, so let's not do anything to prevent it until something truly awful happens.
But that is why we have snapshots and backups and other ways of recovering from the "truly awful" that it is absurd to not do updates.
It's far less about wanting to avoid patches as it is having a culture that truly believes that you don't maintain systems - only fixing them after disaster has struck.
-
This part had me:
As IT we need to stand up and explain that unmaintained systems are already broken, disaster just hasn’t struck yet.
After the Ransomwave, I developed a great script/program that utilizes saltstack/winscp/7zip command line to backup all of the connected windows clients/minions, and pushed the plan to upper management, gave it 3 tests today, and everything works as planned.
The best part when I asked the 3 users if crisis occured and your laptop is unrecoverable this backup that I create can it be usefull, they all said yes that is everything we need.
So yeah, its all now to upper management to allow this or not, I create encrypted and compressed 7z archive on both the NAS and end user machine with all of his important files. With retention plan (WinSCP can do this and delete files based on age) and using HTTPS WebDaV for secure file transfer, to avoid Windows net use hell, you know the stupid limitation per windows user account that he can only connect to 1 mapped drive.
Sadly the plan might fail to be active and enabled, cause with each I.T move I make, upper management sometimes freaks out and thinks I have too much power and I reckon this all cause I am local employee and not international one.
Do you know that today I found out that 7zip+WinSCP have been actively developed for 17 years. my God they are so stable and amazing.
-
I bet I've been using 7zip for that long, in fact!
-
@msff-amman-Itofficer said in If It Ain't Broke, Don't Fix It:
Do you know that today I found out that 7zip+WinSCP have been actively developed for 17 years. my God they are so stable and amazing.
That is really cool. I have used 7-zip forever but had no idea that WinSCP had been developed for so long. Thats a great idea and contribution you had msff-amman-itofficer.
-
Unfortunately I live in that type of culture of not doing maintenance. For context I'm not a super experienced admin like a lot of you, I'm just help support for my campus. So when I first started for this school I quickly saw that any kind of software updates were a big problem and we were way behind. Some people did not have any Windows or application updates in 3 years. So then is when I made a mistake. I asked one of the departments to just put in a work ticket every 3 months so I could keep them updated. They liked it and said everything worked better like Office and various things. Imagine that. After the second round of that I got in trouble with management because nothing was actually wrong yet and there should not have been a work order put in yet. I was told not to do that anymore and just wait until a work ticket came in when there was actually an issue. So that's what i do now.
-
@jmoore said in If It Ain't Broke, Don't Fix It:
Unfortunately I live in that type of culture of not doing maintenance. For context I'm not a super experienced admin like a lot of you, I'm just help support for my campus. So when I first started for this school I quickly saw that any kind of software updates were a big problem and we were way behind. Some people did not have any Windows or application updates in 3 years. So then is when I made a mistake. I asked one of the departments to just put in a work ticket every 3 months so I could keep them updated. They liked it and said everything worked better like Office and various things. Imagine that. After the second round of that I got in trouble with management because nothing was actually wrong yet and there should not have been a work order put in yet. I was told not to do that anymore and just wait until a work ticket came in when there was actually an issue. So that's what i do now.
Probably not worth rocking the boat, but if you ever want to pursue the point, the issue is called "professional negligence." If you have it in writing that they told you not to patch even though they knew there were updated available because they don't consider a security risk to be "broken" that would qualify, in many cases, in court as potentially criminal negligence should a security breach ever occur. That stuff is a big deal. When they violate super basic "entry level" security like that knowingly, they can end up being responsible for exposures. If customer data was to be stolen, they'd have individuals to go after.
-
Oh that's interesting, I didn't know that. Thanks for the information. Yeah I don't want to rock the boat at all but was just surprised by their point of view. I'm personally big on maintaining things at home, digital or otherwise.
-
@jmoore said in If It Ain't Broke, Don't Fix It:
Oh that's interesting, I didn't know that. Thanks for the information. Yeah I don't want to rock the boat at all but was just surprised by their point of view. I'm personally big on maintaining things at home, digital or otherwise.
Yeah... its' a little like hiring a security guard who actively tells people to STOP locking the doors or doing the rounds "until things are stolen". It's basically the same as "make sure things get stolen."
