MSPs the New Hacker Target?



  • Justice Department: Chinese Hackers Hit MSPs

    Following news and rumor reports of MSPs being hacked, especially of their clients getting ransomware attacked, a discussion around MSP security sounds like a great idea.

    It seems like most people in the IT space have some MSP involvement, it's very common for IT pros to do MSP work on the side, even if being an MSP is not your full time job.

    MSPs are a different kind of point of risk to companies as MSPs are a potential single point of attack to breach many companies. Let's talk about where risk can exist, does exist, and what good mitigation strategies are!



  • First thing up: VPNs

    These are way less common than they used to be. But still loads of MSPs use them. A VPN is an "open window" type of thread (meaning air passes between systems) and represents an enormous threat vector. Unlike most MSP attack vectors that require the MSP to be hacked, and then the customers hacked, VPNs often create accidental "open air" from one client to another making not only customers at risk from the MSP, but the MSP at risk to the clients, and clients at risk to other clients!

    Inter-company VPNs need a lot of protection to be used safely, and that level of protection would generally make them useless.



  • User individual user credentials whenever possible, not shared credentials.

    It is so tempting, especially because customers often push for this, to has common credentials for tasks. But this means that leaking creds is easy and maintaining them is hard. Not to mention problems tracking their use. Have users log in as themselves, track them, make them maintain their own creds. Keep creds individualized whenever possible.



  • @scottalanmiller said in MSPs the New Hacker Target?:

    User individual user credentials whenever possible, not shared credentials.

    It is so tempting, especially because customers often push for this, to has common credentials for tasks. But this means that leaking creds is easy and maintaining them is hard. Not to mention problems tracking their use. Have users log in as themselves, track them, make them maintain their own creds. Keep creds individualized whenever possible.

    Both at the MSP and your clients. Each MSP Agent should have an account at the client, with maybe an emergency "if all else fails" shared account.



  • Break glass for emergency shared accounts. Keep the shared account info in a locked safe or even a safety deposit box. Some place that you know if it is accessed.



  • @dafyre said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    User individual user credentials whenever possible, not shared credentials.

    It is so tempting, especially because customers often push for this, to has common credentials for tasks. But this means that leaking creds is easy and maintaining them is hard. Not to mention problems tracking their use. Have users log in as themselves, track them, make them maintain their own creds. Keep creds individualized whenever possible.

    Both at the MSP and your clients. Each MSP Agent should have an account at the client, with maybe an emergency "if all else fails" shared account.

    I'd like to think the client could maintain the emergency account - but I could see some companies where the MSP is the ENTIRE IT department, so there would be no one at the company, save maybe the owner/CEO who could have this - but would likely lose it, etc.



  • @Dashrender said in MSPs the New Hacker Target?:

    @dafyre said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    User individual user credentials whenever possible, not shared credentials.

    It is so tempting, especially because customers often push for this, to has common credentials for tasks. But this means that leaking creds is easy and maintaining them is hard. Not to mention problems tracking their use. Have users log in as themselves, track them, make them maintain their own creds. Keep creds individualized whenever possible.

    Both at the MSP and your clients. Each MSP Agent should have an account at the client, with maybe an emergency "if all else fails" shared account.

    I'd like to think the client could maintain the emergency account - but I could see some companies where the MSP is the ENTIRE IT department, so there would be no one at the company, save maybe the owner/CEO who could have this - but would likely lose it, etc.

    That's actually not a bad idea for the clients that can maintain one.



  • @dafyre said in MSPs the New Hacker Target?:

    @Dashrender said in MSPs the New Hacker Target?:

    @dafyre said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    User individual user credentials whenever possible, not shared credentials.

    It is so tempting, especially because customers often push for this, to has common credentials for tasks. But this means that leaking creds is easy and maintaining them is hard. Not to mention problems tracking their use. Have users log in as themselves, track them, make them maintain their own creds. Keep creds individualized whenever possible.

    Both at the MSP and your clients. Each MSP Agent should have an account at the client, with maybe an emergency "if all else fails" shared account.

    I'd like to think the client could maintain the emergency account - but I could see some companies where the MSP is the ENTIRE IT department, so there would be no one at the company, save maybe the owner/CEO who could have this - but would likely lose it, etc.

    That's actually not a bad idea for the clients that can maintain one.

    It's pretty common to do so. Problem is, the MSP also needs confidence that the account is not used without them knowing.



  • @scottalanmiller said in MSPs the New Hacker Target?:

    @dafyre said in MSPs the New Hacker Target?:

    @Dashrender said in MSPs the New Hacker Target?:

    @dafyre said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    User individual user credentials whenever possible, not shared credentials.

    It is so tempting, especially because customers often push for this, to has common credentials for tasks. But this means that leaking creds is easy and maintaining them is hard. Not to mention problems tracking their use. Have users log in as themselves, track them, make them maintain their own creds. Keep creds individualized whenever possible.

    Both at the MSP and your clients. Each MSP Agent should have an account at the client, with maybe an emergency "if all else fails" shared account.

    I'd like to think the client could maintain the emergency account - but I could see some companies where the MSP is the ENTIRE IT department, so there would be no one at the company, save maybe the owner/CEO who could have this - but would likely lose it, etc.

    That's actually not a bad idea for the clients that can maintain one.

    It's pretty common to do so. Problem is, the MSP also needs confidence that the account is not used without them knowing.

    Need a break glass account.



  • @coliver said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @dafyre said in MSPs the New Hacker Target?:

    @Dashrender said in MSPs the New Hacker Target?:

    @dafyre said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    User individual user credentials whenever possible, not shared credentials.

    It is so tempting, especially because customers often push for this, to has common credentials for tasks. But this means that leaking creds is easy and maintaining them is hard. Not to mention problems tracking their use. Have users log in as themselves, track them, make them maintain their own creds. Keep creds individualized whenever possible.

    Both at the MSP and your clients. Each MSP Agent should have an account at the client, with maybe an emergency "if all else fails" shared account.

    I'd like to think the client could maintain the emergency account - but I could see some companies where the MSP is the ENTIRE IT department, so there would be no one at the company, save maybe the owner/CEO who could have this - but would likely lose it, etc.

    That's actually not a bad idea for the clients that can maintain one.

    It's pretty common to do so. Problem is, the MSP also needs confidence that the account is not used without them knowing.

    Need a break glass account.

    That's what we are discussing, I thought, lol.



  • @scottalanmiller said in MSPs the New Hacker Target?:

    @coliver said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @dafyre said in MSPs the New Hacker Target?:

    @Dashrender said in MSPs the New Hacker Target?:

    @dafyre said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    User individual user credentials whenever possible, not shared credentials.

    It is so tempting, especially because customers often push for this, to has common credentials for tasks. But this means that leaking creds is easy and maintaining them is hard. Not to mention problems tracking their use. Have users log in as themselves, track them, make them maintain their own creds. Keep creds individualized whenever possible.

    Both at the MSP and your clients. Each MSP Agent should have an account at the client, with maybe an emergency "if all else fails" shared account.

    I'd like to think the client could maintain the emergency account - but I could see some companies where the MSP is the ENTIRE IT department, so there would be no one at the company, save maybe the owner/CEO who could have this - but would likely lose it, etc.

    That's actually not a bad idea for the clients that can maintain one.

    It's pretty common to do so. Problem is, the MSP also needs confidence that the account is not used without them knowing.

    Need a break glass account.

    That's what we are discussing, I thought, lol.

    He means literally an envelope with a username & password sealed inside protected by a glass case?



  • @dafyre said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @coliver said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @dafyre said in MSPs the New Hacker Target?:

    @Dashrender said in MSPs the New Hacker Target?:

    @dafyre said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    User individual user credentials whenever possible, not shared credentials.

    It is so tempting, especially because customers often push for this, to has common credentials for tasks. But this means that leaking creds is easy and maintaining them is hard. Not to mention problems tracking their use. Have users log in as themselves, track them, make them maintain their own creds. Keep creds individualized whenever possible.

    Both at the MSP and your clients. Each MSP Agent should have an account at the client, with maybe an emergency "if all else fails" shared account.

    I'd like to think the client could maintain the emergency account - but I could see some companies where the MSP is the ENTIRE IT department, so there would be no one at the company, save maybe the owner/CEO who could have this - but would likely lose it, etc.

    That's actually not a bad idea for the clients that can maintain one.

    It's pretty common to do so. Problem is, the MSP also needs confidence that the account is not used without them knowing.

    Need a break glass account.

    That's what we are discussing, I thought, lol.

    He means literally an envelope with a username & password sealed inside protected by a glass case?

    I mean not literally... but pretty close. Offline user credentials that are stored in a safe location sealed away to ensure the business doesn't have access to them until a time comes where the need to break the seal.



  • @dafyre said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @coliver said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @dafyre said in MSPs the New Hacker Target?:

    @Dashrender said in MSPs the New Hacker Target?:

    @dafyre said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    User individual user credentials whenever possible, not shared credentials.

    It is so tempting, especially because customers often push for this, to has common credentials for tasks. But this means that leaking creds is easy and maintaining them is hard. Not to mention problems tracking their use. Have users log in as themselves, track them, make them maintain their own creds. Keep creds individualized whenever possible.

    Both at the MSP and your clients. Each MSP Agent should have an account at the client, with maybe an emergency "if all else fails" shared account.

    I'd like to think the client could maintain the emergency account - but I could see some companies where the MSP is the ENTIRE IT department, so there would be no one at the company, save maybe the owner/CEO who could have this - but would likely lose it, etc.

    That's actually not a bad idea for the clients that can maintain one.

    It's pretty common to do so. Problem is, the MSP also needs confidence that the account is not used without them knowing.

    Need a break glass account.

    That's what we are discussing, I thought, lol.

    He means literally an envelope with a username & password sealed inside protected by a glass case?

    Can be, but a sealed envelope is enough. Something that has to be "broken and reset" after use.



  • One thing I am shocked many MSPs don't do, which we've done since the first deployment, is secure each Office 365 CSP account (delegated access to each customer through one provider portal) with MFA. In reality, if the MSP was compromised, every customer is then compromised.

    I also witnessed many MSPs not securing their secure password databases with MFA. They secured the front end client application in case a computer was compromised or stolen, but the database itself was wide open.



  • @bbigford said in MSPs the New Hacker Target?:

    One thing I am shocked many MSPs don't do, which we've done since the first deployment, is secure each Office 365 CSP account (delegated access to each customer through one provider portal) with MFA. In reality, if the MSP was compromised, every customer is then compromised.

    I get the value of MFA. But how would each customer get compromised if the MSP was compromised in an Office 365 context?



  • @scottalanmiller said in MSPs the New Hacker Target?:

    @bbigford said in MSPs the New Hacker Target?:

    One thing I am shocked many MSPs don't do, which we've done since the first deployment, is secure each Office 365 CSP account (delegated access to each customer through one provider portal) with MFA. In reality, if the MSP was compromised, every customer is then compromised.

    I get the value of MFA. But how would each customer get compromised if the MSP was compromised in an Office 365 context?

    wouldn't the hacker then have access to all of the customer accounts via the MSP's O365 delegate account?



  • @Dashrender said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @bbigford said in MSPs the New Hacker Target?:

    One thing I am shocked many MSPs don't do, which we've done since the first deployment, is secure each Office 365 CSP account (delegated access to each customer through one provider portal) with MFA. In reality, if the MSP was compromised, every customer is then compromised.

    I get the value of MFA. But how would each customer get compromised if the MSP was compromised in an Office 365 context?

    wouldn't the hacker then have access to all of the customer accounts via the MSP's O365 delegate account?

    That's assuming you are using a shared account that can access all customers, rather than a discrete account per customer.



  • @scottalanmiller said in MSPs the New Hacker Target?:

    @Dashrender said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @bbigford said in MSPs the New Hacker Target?:

    One thing I am shocked many MSPs don't do, which we've done since the first deployment, is secure each Office 365 CSP account (delegated access to each customer through one provider portal) with MFA. In reality, if the MSP was compromised, every customer is then compromised.

    I get the value of MFA. But how would each customer get compromised if the MSP was compromised in an Office 365 context?

    wouldn't the hacker then have access to all of the customer accounts via the MSP's O365 delegate account?

    That's assuming you are using a shared account that can access all customers, rather than a discrete account per customer.

    Of course.

    So what does NTG do?



  • @Dashrender said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @Dashrender said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @bbigford said in MSPs the New Hacker Target?:

    One thing I am shocked many MSPs don't do, which we've done since the first deployment, is secure each Office 365 CSP account (delegated access to each customer through one provider portal) with MFA. In reality, if the MSP was compromised, every customer is then compromised.

    I get the value of MFA. But how would each customer get compromised if the MSP was compromised in an Office 365 context?

    wouldn't the hacker then have access to all of the customer accounts via the MSP's O365 delegate account?

    That's assuming you are using a shared account that can access all customers, rather than a discrete account per customer.

    Of course.

    So what does NTG do?

    Individual accounts per customer. We aren't a reseller, so there isn't any natural connection between customers already.



  • @scottalanmiller said in MSPs the New Hacker Target?:

    @Dashrender said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @Dashrender said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @bbigford said in MSPs the New Hacker Target?:

    One thing I am shocked many MSPs don't do, which we've done since the first deployment, is secure each Office 365 CSP account (delegated access to each customer through one provider portal) with MFA. In reality, if the MSP was compromised, every customer is then compromised.

    I get the value of MFA. But how would each customer get compromised if the MSP was compromised in an Office 365 context?

    wouldn't the hacker then have access to all of the customer accounts via the MSP's O365 delegate account?

    That's assuming you are using a shared account that can access all customers, rather than a discrete account per customer.

    Of course.

    So what does NTG do?

    Individual accounts per customer. We aren't a reseller, so there isn't any natural connection between customers already.

    What does a natural connection between customers have to do with anything?

    a single vendor account with MS which then grants you access to ALL of your customers accounts, prevents you from needing to log in dozens of times a day - from having to maintain all those separate accounts, etc.

    of course, it opens you up to the above stated issues.



  • @Dashrender said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @Dashrender said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @Dashrender said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    @bbigford said in MSPs the New Hacker Target?:

    One thing I am shocked many MSPs don't do, which we've done since the first deployment, is secure each Office 365 CSP account (delegated access to each customer through one provider portal) with MFA. In reality, if the MSP was compromised, every customer is then compromised.

    I get the value of MFA. But how would each customer get compromised if the MSP was compromised in an Office 365 context?

    wouldn't the hacker then have access to all of the customer accounts via the MSP's O365 delegate account?

    That's assuming you are using a shared account that can access all customers, rather than a discrete account per customer.

    Of course.

    So what does NTG do?

    Individual accounts per customer. We aren't a reseller, so there isn't any natural connection between customers already.

    What does a natural connection between customers have to do with anything?

    There is no association between the customers, even at the ITSP level. No natural reason for any cross connection to exist.



  • @Dashrender said in MSPs the New Hacker Target?:

    a single vendor account with MS which then grants you access to ALL of your customers accounts, prevents you from needing to log in dozens of times a day - from having to maintain all those separate accounts, etc.

    of course, it opens you up to the above stated issues.

    I'm not saying that it is a bad thing, just not one that we use.



  • @scottalanmiller said in MSPs the New Hacker Target?:

    @Dashrender said in MSPs the New Hacker Target?:

    a single vendor account with MS which then grants you access to ALL of your customers accounts, prevents you from needing to log in dozens of times a day - from having to maintain all those separate accounts, etc.

    of course, it opens you up to the above stated issues.

    I'm not saying that it is a bad thing, just not one that we use.

    Cool -



  • Literally on the phone with the customer of a different MSP that had this happen.



  • @scottalanmiller said in MSPs the New Hacker Target?:

    Literally on the phone with the customer of a different MSP that had this happen.

    Is NTG reaching out to these MSPs to offer assistance and/or guidance?



  • @Obsolesce said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    Literally on the phone with the customer of a different MSP that had this happen.

    Is NTG reaching out to these MSPs to offer assistance and/or guidance?

    Or are they reaching out to customers to offer competent managed services?



  • @RojoLoco said in MSPs the New Hacker Target?:

    @Obsolesce said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    Literally on the phone with the customer of a different MSP that had this happen.

    Is NTG reaching out to these MSPs to offer assistance and/or guidance?

    Or are they reaching out to customers to offer competent managed services?

    Yeah that too!



  • @Obsolesce said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    Literally on the phone with the customer of a different MSP that had this happen.

    Is NTG reaching out to these MSPs to offer assistance and/or guidance?

    We always offer that service. MSPs are free to reach pur to us always.



  • @RojoLoco said in MSPs the New Hacker Target?:

    @Obsolesce said in MSPs the New Hacker Target?:

    @scottalanmiller said in MSPs the New Hacker Target?:

    Literally on the phone with the customer of a different MSP that had this happen.

    Is NTG reaching out to these MSPs to offer assistance and/or guidance?

    Or are they reaching out to customers to offer competent managed services?

    That would be smart. But knowing who affected customers are is hard.



  • MSP Maturity Model. Strictly speaking, the MSPMM does not tell MSPs to make all of their customers identical. But in practice, it encourages it and many MSPs talk about the MSPMM in these terms - finding ways to make customers all run the same tools, software, practices, network design, etc. This makes management so much easier for the MSP, but has two major problems.

    First, it forces the customer to conform to the vendor, which makes very little sense. IT needs to adapt to the business, not the business to IT. But that's another topic.

    Secondary, it means that an attack vector that works on the MSP will likely work on every single one of their customers making the prospect of breaching the MSP that much better. Sure, if a targeted attack by experienced state-sponsored hackers goes after an MSP, the MSP has little chance of winning that battle. But that isn't the real risk. In the real world, the risk is automated attacks looking for common vulnerabilities and spreading organically through shared tooling - things that are only possible or reasonably likely when the environments are homogeneous: both amongst the MSP clients, and between clients and the MSP themselves.

    The traditional approach of MSPs, especially VAR - MSP combo companies, is to have not only the same tools and software, but even the same hardware and products so that any hole anywhere because a hole everywhere and breaching any one piece of the infrastructure means you are likely to breach it all.


Log in to reply