Is Most IT Really Corrupt?
-
@dashrender In my mind, a bench tech actually does tech work. Like troubleshooting issues that can't be done remotely, like a workstation that won't boot. You're not troubleshooting that from a few hundred miles away very well lol, someone has to put hands on that to do anything with it. It's difficult to maintain cost effectiveness except in particularly good SMBs with multiple spares sitting on site, and it takes more time to do a workstation swap managed by a clueless user versus a legitimate technician of just about any level. If they have any issues at all, it's burning vastly more expensive MSP time at over $100/hour minimum for direct support that isn't covered in a contract.
In my agency, we have about 40 printers on desks, and we buy them because it was enormously expensive to try and pay for those to be supported (though our large multi-functions are all rented and contracted for support from a printer company). Sure the users can change paper and cartridges themselves.. and that's about it though, lol. They don't understand enough about printers to troubleshoot them (it's not their job, which I can understand to a point). Outside of IT, there are exactly zero people in my organization that would be remotely comfortable disassembling part of a printer to replace parts. Desk printers suck, and they're tedious as hell to support, or expensive as hell in our experience. Certainly not necessary in many organizations, but even without highly paid staff, it's just WAY too expensive to try and funnel that much printing through a few big printers.
Also, I'm early 30s and I've been at my current org for about five years @Wirestyle22.
For me, benefits don't cost us anywhere remotely close to 25K/yr, it's more like 15K/yr. My organization was paying about 60k/yr rate for maintenance and management of the entire environment between my predecessor and when I was employed, and that was the best option available from any MSP within 150 miles. The month I started, we had a major server failure (host failure) that nearly shut the company down for a full week and change. We quoted costs for having MSPs help us, it was (at the time) almost a full year's pay for me to pay for the least expensive MSP offer we got to resolve one major failure. It turns out, there were far better MSPs available, and they all cost substantially more to contract than what I'm paid to do what I do, and our cost savings are still much greater yet for MSPs to do what I have from a project standpoint in rebuilding the entirety of our IT infrastructure over the past few years.
My organization still requires a security officer (I'de argue that honestly, most SMBs past a very small size really should have some sort of security post of some form or other), so they would need to pay like 80%-100% of my salary to someone regardless.. but we as an SMB save about 10K-20K per year on IT after hiring a second individual for our internal IT by not hiring an MSP. The MSP argument only makes obvious sense if you ignore all the other roles that IT makes sense to do in an SMB. Reality often doesn't bare that kind of division of roles and responsibilities out well, so it's kind of a theoretical benefit in many cases. That however, isn't to say that I don't agree that a lot of SMBs really should be contracting out all of their IT, my argument from the beginning has been that it's just bad advice to suggest that Every SMB should contract an MSP because it's cheaper/better etc., because it quite simply isn't the case. Being an exception to the norm helps make that more apparent of course, and I realize I'm an exception in a myriad of ways. I realize though that I'm far from the only one, and my issue is only really with the argument using the word "always".
If my organization could get better benefits from an MSP, it's the MSPs fault for not making themselves easily accessible enough for my organization to detect them. I doubt there are or would be many that could offer nearly the value they receive from me, but I'm certain some MSPs might be capable of offering superior results in some ways... but at how much greater cost? We've basically spent about 340k to manage and administer the environment for about five years, while replacing and upgrading everything from top to bottom (literally, all the things) for about 50 employees including the physical hosts, server software, switches, wiring, phone system, all workstations, our entire environment's implementation as well as how and with what we do our security. There are few MSPs using what we do at my organization, which is fine. However, they're taking advice from me more often than the opposite where I am. Again, I realize I'm very much an anomaly though.
-
@tirendir said in Is Most IT Really Corrupt?:
it's just WAY too expensive to try and funnel that much printing through a few big printers.
What? How is it more expensive to use fewer larger more efficient/less cost per page printers than putting a printer on every desk?
-
@tirendir said in Is Most IT Really Corrupt?:
The month I started, we had a major server failure (host failure) that nearly shut the company down for a full week and change. We quoted costs for having MSPs help us, it was (at the time) almost a full year's pay for me to pay for the least expensive MSP offer we got to resolve one major failure.
Holy cats, what was the issue? Let's assume the MSP wanted $150/hr, you mentioned the previous MSP cost the company $60K for a year, so we'll use that for the yearly cost.... The MSP said it would take 400 man hours to fix your server? Something seems really wrong with that number.
Also - I'm wondering, did you look for an ITSP (hourly consultant) to work on the server instead of a long term contract MSP to fix it? Also - did the fix include new hardware that was needed regardless? If so, that money doesn't count in the cost. And lastly - what did you end doing to fix it?
-
@dashrender Time costs money, and forcing 40 people to two or three printers costs a LOT of time when they're only printing five or six pages each, and they're all different pages for difference cases etc.. It's a time cost problem, not an ink or toner cost problem.
-
@tirendir said in Is Most IT Really Corrupt?:
@dashrender Time costs money, and forcing 40 people to two or three printers costs a LOT of time when they're only printing five or six pages each, and they're all different pages for difference cases etc.. It's a time cost problem, not an ink or toner cost problem.
Huh - it's a time cost problem you say - lol, unless they are printing 4-5 pages like every few mins I don't see the issue. On a typical desktop printer, a page costs $0.25/ea or more. So those pages cost a $1 or more.
Granted I have to include the cost of printed pages from a big printer, My Konica department printer costs about $0.02/page all costs included (cost of machine/cost of paper/electricity/maintenance, etc) I don't think the $0.25 included all costs, but lets assume it does, that's still $0.23/page more on the desktop printers.So, how much time did the user spend walking to the printer?
Even at $20/hr, the user would need to spend 3 mins walking to/fro to make the cost roughly the same.
-
@dashrender said in Is Most IT Really Corrupt?:
So, how much time did the user spend walking to the printer?
Even at $20/hr, the user would need to spend 3 mins walking to/fro to make the cost roughly the same.
Hey they could be handicapped. . .
-
@dashrender The fix would have run us some 10s of thousands of dollars, because the Host was setup on RAID5 with four drives prior to my arrival. We had a triple-drive failure on that Host, and the backups they were supposedly managing never actually happened off-site, and the on-site backups it turned out were scrambled so badly by the box they were being sent to on-site, that they were completely un-readable, meaning that the only fix they could offer was essentially a complete software-side environment rebuild from the ground with an on-site exchange orphaned from it's domain in that case.
The problem was largely terrible configuration, and the MSP never bothered to even attempt to do anything about it, advise on what a lousy config it was, or anything of the sort. The only reason we were able to avoid an essentially catastrophic trainwreck was because it so happened that I was in week two of being there, and I had setup a Server 2012R2 VM to tinker with in the environment. I had been working on putting it together as a replacement for the existing, sloppy file server we had that showed obvious attempts at reigning in the file sprawl and instituting better organization at least twice without success. Rather than try and go a third time on a Server 2003 VM that had had this attempted twice, I thought it would be smarter to setup a 2012R2 VM that we already owned licensing for. I ended up basically getting our environment functional enough to get by for a week by being creative while rebuilding the rest of the environment in pieces since the structure was still 2000 architecture, PDC and all (which was on the failed host and nowhere else).
At the time, my org was offering me 36K/yr, because I had zero Admin experience whatsoever, although I had well over a decade of IT experience. The quote we got was crazy. Suffice to say, we ditched that provider, and they were bought out not long after with little staff retention. In business terms, they were taking a significant risk on me with no Admin experience, but it turned out very much in their favor. I've been getting pay raises every year plus merit raises since I started, which has gotten me up to where I am now. Enterprises wouldn't give me a chance, but that's a different topic altogether.
-
@dashrender said in Is Most IT Really Corrupt?:
@tirendir said in Is Most IT Really Corrupt?:
@dashrender Time costs money, and forcing 40 people to two or three printers costs a LOT of time when they're only printing five or six pages each, and they're all different pages for difference cases etc.. It's a time cost problem, not an ink or toner cost problem.
Huh - it's a time cost problem you say - lol, unless they are printing 4-5 pages like every few mins I don't see the issue. On a typical desktop printer, a page costs $0.25/ea or more. So those pages cost a $1 or more.
Granted I have to include the cost of printed pages from a big printer, My Konica department printer costs about $0.02/page all costs included (cost of machine/cost of paper/electricity/maintenance, etc) I don't think the $0.25 included all costs, but lets assume it does, that's still $0.23/page more on the desktop printers.So, how much time did the user spend walking to the printer?
Even at $20/hr, the user would need to spend 3 mins walking to/fro to make the cost roughly the same.
40 people vs 2 printers (not all of the employees need to print like that) = a lot of not able to print at the same time, or a lot of standing around with not so trustworthy clients in their offices where their purses, phones, etc. are. I don't know if any of you deal with the public a lot or not, but our clientele are not to be trusted, as they have stolen from many of our employees before. Also, the cost for laser prints isn't remotely $0.25/print (we do use laser printers), though it's absolutely more than a big multifunction, I concur. We tried consolidating printers, it wasted a lot of time and money, because wasting 2 minutes a person per day X 40 X 5 X 52 = a lot of money wasted in useless labor. Only a little shy of 350 hours a year in wasted labor money. Not even remotely worth considering worth it in any way, because that's thousands of dollars in labor in a year that is 100% wasted.
-
@tirendir said in Is Most IT Really Corrupt?:
@storageninja I'm saying that if there were enough good MSPs, I would agree with you. But there simply aren't enough of them, which means many of the realistic options for MSPs for most organizations aren't much if any better than the lousy SMB IT Scott's been talking about for a while. Reality doesn't realize a world where good MSPs are available to everyone in practical terms. The good MSPs aren't large enough, plentiful enough, or spread out well enough to service the vast majority of organizations that need good IT services is how it seems to work out ultimately. That's the reason there is so much meh or even poor IT going on, imo.
This is kind of a weird fallacy because MSP's can cover a LOT of ground without a lot of staff. (Not that there are not some massive ones like All Covered) as well as tons of medium ones like RoundTower etc. I worked for a small one in Houston and we had no problem support customers in EMEA (It was easier). We had a local resource deal with end user support (What little there was left), and all the back end maintenance stuff was easy to do as our day was there after hours. having a contract model where that literally forces them from adopting stupid things (no PST's allowed was my favorite one) ended up with a far more value and transparency for the IT spend than purely in house.
A MSP doesn't have to be in the same zip code to cover a client. If it does, they are honestly a shitty MSP.
-
@tirendir said in Is Most IT Really Corrupt?:
@dashrender The fix would have run us some 10s of thousands of dollars, because the Host was setup on RAID5 with four drives prior to my arrival. We had a triple-drive failure on that Host, and the backups they were supposedly managing never actually happened off-site, and the on-site backups it turned out were scrambled so badly by the box they were being sent to on-site, that they were completely un-readable, meaning that the only fix they could offer was essentially a complete software-side environment rebuild from the ground with an on-site exchange orphaned from it's domain in that case.
What's great about this story is it has nothing to do with a MSP, or a MSP model. In house IT can not pay attention to backups, or setup everything in RAID 0 (Working at a MSP we had to clean this up a lot, when onboarding clients).
-
@tirendir said in Is Most IT Really Corrupt?:
@dashrender In my mind, a bench tech actually does tech work. Like troubleshooting issues that can't be done remotely, like a workstation that won't boot. You're not troubleshooting that from a few hundred miles away very well lol, someone has to put hands on that to do anything with it. It's difficult to maintain cost effectiveness except in particularly good SMBs with multiple spares sitting on site, and it takes more time to do a workstation swap managed by a clueless user versus a legitimate technician of just about any level. If they have any issues at all, it's burning vastly more expensive MSP time at over $100/hour minimum for direct support that isn't covered in a contract.
We did VDI a lot for service delivery. Thin clients REALLY don't die much. Eliminated the need for bench entirely, and spares were cheap to have sitting in each branch office.
-
@tirendir said in Is Most IT Really Corrupt?:
For me, benefits don't cost us anywhere remotely close to 25K/yr, it's more like 15K/yr.
There's a lot of costs to carrying an employee. Unemployment insurance/tax, the other 1/2 of FICA, corporate insurance and liability often is a per head charge and then others that could be sunk costs, or not (Real estate for office, air con/power for employee, parking spots that can be anything from a free dirt pad, to a $150 a month contract garage). Throw in software licensing (CAL's, 365/office) and general software licensing for IT specific tools (License for Fusion, a Secret server for password management) and the stuff adds up. MSP's get to amortize this stuff across multiple customers and their RMM and other tools get to take advantage of licensing at scale. We paid maybe $2 per device per month we were doing log analytics against, where a comparable onsite license for the same product was $250 per device. (We didn't even have much scale).
-
@dashrender said in Is Most IT Really Corrupt?:
One of the things here is not to simply look at cost savings, but cost parity. If the company gets all the same work done at a higher quality at the same cost, that's a win.
When I worked for a MSP, no one came to us to save money. (and I mean no one).
They came to us over money but it boiled down to...
- They wanted to get more value for the same spend.
- They wanted to spend more, but know where it was going.
I'd argue SaaS business have the same model. I can keep using my 12-year-old copy of ACT for CRM, and it's a TON cheaper than SalesForce (Sunk cost) but I see a hell of a lot of companies going to SalesForce.
If you think your primary job in IT is to control cost (and not to increase agility for the business, or mitigate risk) then at best you're going to be stuck at 30% below market rate for someone with your number of years in IT, and at worse you'll get pushed out by further automation, SaaS/Cloud/Simplification.
-
@dashrender said in Is Most IT Really Corrupt?:
As far as job sharing, that's precisely why I said that SMBs require generalists. There will likely never be a time or place where SMBs don't need generalists (not just in IT), and an IT generalist is a cost-saving factor that no one who has posted here has seemed to enter into their calculations. If you try and consider SMBs in Enterprise terms, you're quite likely doing it wrong, because SMBs aren't generally like that until they reach a certain size. They're entirely different types of entities whose similarities largely end at the point where you acknowledge that they are businesses of different scale. You can't run an SMB like an Enterprise, because SMBs don't possess the scale to staff like an Enterprise. It's extremely rare in most fields that it is cheaper to hire a contractor than it is to utilize your own staff. The cost-savings only ever comes when expertise is required, or regulation requires typically. The on-staff IT in an SMB is probably not only doing IT, you're correct. That is precisely why the MSP saves money argument is wonky, because it fails to account for the other tasks that that in-house IT staff does in addition to the basic management and maintenance of the environment. Sure, one can make the argument that the IT staff may not be adequately managing the environment... but it's a may, not a will or a must. It's also entirely possible that they're bringing lots of added value over an MSP by doing other tasks that an MSP won't do at all, or will only do for added, cumulative costs that simply don't exist with on-site staff.
Of course this is true - but it's now likely that you are over or under paying that person for these other non IT jobs. It's extremely unlikely that a person is clocking out when done with IT work at wage X and then clocking in at wage y when they are doing non IT things.
This is where the company can either come out ahead or behind.
So we have two situations, the over and the under paid - there's always the potential for parity pay, but that really seems pretty unlikely based on skill sets.You either end up with someone who's bad at IT and the right price for taking apart printers or mopping the floors, or you get someone who's amazing at IT and completely over priced.
In reality, you get the former until they leave, get replaced by the latter, suffer from thisuntil they become the former, they leave and the cycle starts all over again.
-
@tirendir said in Is Most IT Really Corrupt?:
40 people vs 2 printers (not all of the employees need to print like that) = a lot of not able to print at the same time, or a lot of standing around with not so trustworthy clients in their offices where their purses, phones, etc. are. I don't know if any of you deal with the public a lot or not, but our clientele are not to be trusted, as they have stolen from many of our employees before. Also, the cost for laser prints isn't remotely $0.25/print (we do use laser printers), though it's absolutely more than a big multifunction, I concur. We tried consolidating printers, it wasted a lot of time and money, because wasting 2 minutes a person per day X 40 X 5 X 52 = a lot of money wasted in useless labor. Only a little shy of 350 hours a year in wasted labor money. Not even remotely worth considering worth it in any way, because that's thousands of dollars in labor in a year that is 100% wasted.
Where are you working that you have the public directly having that type of access? and if that's really the case, why not have a lockable drawer in every office, problem solved.
That time 350 hours, they make what, $20/hr? = $7000 How much is spent supporting those 40 printers vs 3 or 4 printers? what's the cost difference between 40 laser printers vs 3-4 large ones? waiting around? get faster printing printers.
There are places and reasons to deploy individual printers, but I haven't heard them in this case yet.
-
@dashrender said in Is Most IT Really Corrupt?:
or a lot of standing around with not so trustworthy clients in their offices where their purses, phones, etc. are. I don't know if any of you deal with the public a lot or not, but our clientele are not to be trusted, as they have stolen from many of our employees before
I like central print servers where you walk up, enter your code, then it dumps the print queue for what you had queued (on some crazy fast printer). You can do secure printing in a central manner this way while still maintaining compliance. It also has the benefit of they can pick up their print jobs from ANY office.
-
@storageninja said in Is Most IT Really Corrupt?:
@dashrender said in Is Most IT Really Corrupt?:
or a lot of standing around with not so trustworthy clients in their offices where their purses, phones, etc. are. I don't know if any of you deal with the public a lot or not, but our clientele are not to be trusted, as they have stolen from many of our employees before
I like central print servers where you walk up, enter your code, then it dumps the print queue for what you had queued (on some crazy fast printer). You can do secure printing in a central manner this way while still maintaining compliance. It also has the benefit of they can pick up their print jobs from ANY office.
your quote is broken again, I didn't say that.
-
@storageninja said in Is Most IT Really Corrupt?:
@scottalanmiller said in Is Most IT Really Corrupt?:
@tirendir said in Is Most IT Really Corrupt?:
@scottalanmiller Totally agree it's often really obvious in SMB, but to them it's not a question of whether there is theft or not even when they notice it, but whether the individual is worth losing over it. Enterprise minded people have this mistaken idea oftentimes that everyone is replaceable or interchangeable. SMBs don't have the luxury of such a silly notion, so when they detect theft they must weigh relative value where Enterprises often simply don't bother because they seem to think they don't have to.
SMBs definitely have that option. It's an illusion that the do not. SMBs need fewer resources and at a lower level so actually have more ability to replace. What's often approaching impossible for an enterprise might be trivial for an SMB. SMB needs are so often generic and interchangeable compared to enterprise. They have a big advantage here.
Wait, is he saying people in SMBs are not easy to replace? The MSP and SaaS industry continue to make that a less defendnable position.
I think so. But SMB is so easy to replace. There is a waiting line of out of work or looking for different work SMB people (all with experience) for every available position out there. No one lacks someone for an SMB position. And nearly anyone from the enterprise can do SMB work, might be a small learning curve, but most SMB is much easier than most enterprise work (always exceptions) so there is a pretty big pool available there, too. And if SMBs were willing to pay market rates, they could hire people out of MSPs, too.
Because SMBs have the lowest requirements for staff, they have the easiest time hiring. It just makes the least sense to do so.
-
@storageninja said in Is Most IT Really Corrupt?:
@tirendir said in Is Most IT Really Corrupt?:
As far as underhanded or shady purchasing deals with kickbacks, I would agree that IT are pretty uniquely situated to participate in such practices far more so than the vast majority of fields. I'll also agree that SMBs get the short end of things in terms of quality personnel of course, because they don't have the scope of reach for talent recruitment, nor the vast resources that Enterprises typically do, so oftentimes the Enterprises will scoop up much of the best talent before SMBs ever get a chance. Such scenarios obviously would leave the SMBs with far less comparable or adequately capable talent to choose from, forcing them to have to make due with what they have left to select from. Ironically, the biggest issue with SMBs may well be Enterprises gobbling up much of the best talent, perhaps as much as the fact that SMBs may not be great businesses.
A huge issue with SMB's is they have far less robust hiring practices. SMB's tend to cheap out on background checks for criminal actions (and in some cases even hilariously waste money on drug tests for IT staff, while using the "budget" background check).
I saw an IT director wiretap a board meeting at a SMB once. The guy was a little off, but honestly, I blame management. They wanted to have an IT director slash custom software developer who worked 70 hour weeks with 1 week of vacation a year and they paid a farcical 100K with no variable comp. If you have such unrealistic compensation requirements, your only option is going to be getting someone who's an idiot or worse, has some "fun" quirks like ethics or mental health.
Or, quite often, is totally unqualified AND has some quirks. The two often go together.
-
@storageninja said in Is Most IT Really Corrupt?:
@scottalanmiller said in Is Most IT Really Corrupt?:
This was an interesting read. I had never really given any thought to corruption in IT beyond the faking it idea. I sure as heck wouldn't feel comfortable working somewhere and not knowing what I'm doing and not even making an effort to learn.
And yet, it's so common that people no longer even realize it is something to question!
As long as you know how to learn quickly this one is often over-rated. It's the guys who NEVER learn what their job is (20 years in, having the SE come in once a week and make changes to network for them in exchange for buying enterysys switches) where it's out of hand.
That's what we see so much of. People doing the same job for a decade and not knowing even the most basic things about their specific jobs or environments. Outsourcing everything to the point where their own job has no point - a secretary could do it in 30 minutes a week without needing any training. We find this a lot. People just cashing a paycheck when they've actually outsourced themselves but told no one.
