When a C-level gets the boot



  • I had an interesting event happen about a year ago, and I'm wondering how more effectively I could have handled this one. Here are some of the high points:

    • I take on a new account, a global aerospace company.
    • They are using a mix of Windows and Linux systems.
    • The CEO of that company is an enterprise admin. Why they have this or where they find the time to do anything on the network without breaking lots of things is beyond me. I asked about it, but was never given a valid response; moving on...
    • The CEO made some bad choices, and was terminated in spectacular fashion.
    • I was not told before hand that the CEO was going to be fired. I wasn't actually even told for about a week they were gone. I had finally emailed the CTO after the CEO stopped responding to messages (no forwarding was setup on their account, because I was never told).
    • The CEO still has remote access, and has created some service accounts that keep being added to enterprise admin security group.
    • Eventually figured out the CEO was behind this, after looking into some audit logs.
    • Company's legal team sent out a cease-and-desist.
    • I change the passwords on all the service accounts possibly related to him, disable all those accounts, and do the same for his own domain account.
    • Removed him from all security groups, as well as the service accounts. In hopes this would at least slow him down if he still had some kind of access.
    • Created more specific legal banners.

    After that, it was basically a dance of disabling accounts, looking at account auditing logs and setting up alerting about new accounts being created/changed/etc.

    Here's the question... what more would you be doing if you were in that situation, dealing with someone vindictive and had that level of access to begin with?



  • @bbigford said in When a C-level gets the boot:

    Here's the question... what more would you be doing if you were in that situation, dealing with someone vindictive and had that level of access to begin with?

    Nothing more. It is in the hands of legal at that point.

    From a technology perspective, your system is compromised. No different than an outside hacker. The only way to know you are clean is to nuke it from orbit, or spend an insane amount of time and money auditing every single file and every single machine on the entire network.



  • That's a question to ask of the board.... why was the CEO allowed to retain access after being fired? Why did his replacement keep something so important a secret from the security people? Sounds like there was something much more wrong than jut the CEO himself. That should trigger a security audit of things much deeper than what was being looked at.



  • @scottalanmiller said in When a C-level gets the boot:

    That's a question to ask of the board.... why was the CEO allowed to retain access after being fired? Why did his replacement keep something so important a secret from the security people? Sounds like there was something much more wrong than jut the CEO himself. That should trigger a security audit of things much deeper than what was being looked at.

    Not relevant to the question at hand. Mistakes were made, that is obvious. But those are postmortem issues at this point. Not remediation issues.



  • Simple answer from what you could do more, nothing.

    You found out after the person was terminated, and closed off everything you knew about. As @JaredBusch said, nuke it from orbit is the only "true" way to be 100% certain that the ex-employee is off of the system.

    This of course brings challenges, and should be left to legal to deal with before nuking everything.



  • I agree, you have to nuke the network from orbit. You "are" and were compromised for some time. No telling how deep it goes.



  • @scottalanmiller said in When a C-level gets the boot:

    I agree, you have to nuke the network from orbit. You "are" and were compromised for some time. No telling how deep it goes.

    Exactly.

    And you earlier comments are true. The post mostmortem should turn up a number of things that mean the board screwed the pooch in a number of ways, as well as a few other C levels. This could trigger action by the SEC if it is a traded company.



  • @jaredbusch said in When a C-level gets the boot:

    @scottalanmiller said in When a C-level gets the boot:

    I agree, you have to nuke the network from orbit. You "are" and were compromised for some time. No telling how deep it goes.

    Exactly.

    And you earlier comments are true. The post mostmortem should turn up a number of things that mean the board screwed the pooch in a number of ways, as well as a few other C levels. This could trigger action by the SEC if it is a traded company.

    There were a lot of postmortem things that needed to change. I could have nuked more from orbit, but with a global company and around 30k employees, I just needed more time. I left the company a couple months later for unrelated reasons (I hated it there in general), so I never got to dealing with the tailings.


  • Vendor

    Couple things...

    1. At 20K users you should have a dedicated SOC or an outsource SOC doing 24/7 analytics of the logs and logs should be going somewhere IMMUTABLE (LogLogic etc).

    2. If someone who is fired is creating accounts you need to call local law enforcement and refer this to them.

    3. Track the time and labor involved in the cleanup. Bring in an outside security audit firm. If this crossed state lines or other factors on the cost of remediation this may involve the FBI.

    4. If this is a public company the EXTERNAL accounting auditors need to be notified of the lack of internal controls. There may be SEC violations if policies didn't exist that need to, or were not followed that did exist. Going to lunch with external auditors and telling them what was fucked up was a GREAT way as a consultant to make sure a fire got lit for someone to fix something.

    5. If significant fraud or other things are found call the SEC directly and report. Whistleblowers get paid well.



  • @bbigford said in When a C-level gets the boot:

    • The CEO of that company is an enterprise admin. Why they have this or where they find the time to do anything on the network without breaking lots of things is beyond me. I asked about it, but was never given a valid response; moving on...

    As a consultant, why would you not make a bigger deal out of that?



  • @irj said in When a C-level gets the boot:

    @bbigford said in When a C-level gets the boot:

    • The CEO of that company is an enterprise admin. Why they have this or where they find the time to do anything on the network without breaking lots of things is beyond me. I asked about it, but was never given a valid response; moving on...

    As a consultant, why would you not make a bigger deal out of that?

    As a consultant, you only want to bring it up. Let them determine the level of deal that it is for them.


Log in to reply