The argument for official support vs third party support
-
So @StorageNinja brings this up quite often, that you need paid support from the software developer. Especially for VMWare's ESXi.
He then states that third party support isn't sufficient for issues like and I quote "In all cases for us to support you, we required you have a support agreement with Citrix, or VMware as when we hit bugs (That required CPD engineering teams to write code) that was WAYYYY out of scope of what a MSP does or provides."
So my question is, why is VMWare's ESXi constantly recommended, if bugs are so prolific that end users and SMB's are finding bugs and needing patches to it directly from the developer; who hasn't yet discovered said bug.
Why is official support from Citrix or third party certified people not sufficient or frowned upon when using XenServer?
-
Is support from VmWare so amazing that every time you have found a bug you leave with a smith on your face and $20 poorer. . .
Something isn't adding up.
I get that bugs exist in all software, totally do. I even understand that bugs last for a long time on a lot of platforms (smbv1 bug from Microsoft) or the list of publicly displayed bugs for XenServer that are open for a long period of time.
What doesn't make sense is why push and promote a solution for the case of "well if you find a bug in the software you can get it fixed from the developer"
Who in my opinion should already be fixing bugs if they are so critical that they'll break entire systems.
-
I have to agree, if the premise is that people keep choosing software that doesn't work under many conditions and the vendor doesn't fix when a bug is found... sounds like software to generally avoid. If you were to use RHEL without vendor support, they would still patch their systems and fix known bugs. You might not get top priority if you are the only person being affected by a bug, but in the real world, this really doesn't happen, especially in the SMB. Microsoft doesn't even offer support in the SMB, so this logic implies that you can't use MS products across the board because MS doesn't issue a vendor support agreement upon purchase (you can buy support case by case, but if they don't want to fix the issue, they just don't charge you.)
Same with Ubuntu LTS, find a bug and you have support and they don't feel it is worth fixing, they just tell you to move to a version that they will fix or refund your money presumably. Vendor support isn't magic, they are the ones introducing the problem needing fixing in the first place, anyway.
In the real world, I can go decades without needing vendor support. That's not to say that vendor support is bad, it's a great thing. But in three decades of IT I've needed to call a vendor for OS support (including Windows) a total of zero times. I've been involved with someone that did have vendor support, called them, and was told that they would not get support (both Ubuntu and Windows) but never in a position where vendor support was needed and provided.
I'd argue that third party support, in the real world, ends up being vastly more valuable, especially in the SMB where vendor support is generally anemic at best and there is no leverage to ensure that good support is provided. Broad support from a good service provider that can not only support the product, but can also offer alternatives, is way more useful and important than giving more money to the vendor with the problem and then being completely dependent on them to fix things and to fix them in a way that you like.
-
I think this is one of those places where enterprise needs and options are different from the SMB. If you are an SMB and pay for support, you may or may not get support. It's a crap shoot, especially with big, well known software. You are so small, they'll just drop you as a customer. It's cheaper to refund your money or face you in court than to fix bugs only you see. They know you can't do anything about it. But if you are a Fortune 500 and have a legal team and a rock solid contract that says that they have to support you or pay huge penalties and you can drag them through the media for having worthless products, you get awesome support, because you have leverage.
This is a case where market realities come to bare and make a big difference. In a small company, you also almost never hit the bugs and edge case issues that the big companies do. You are not likely using special hardware or software, pushing the envelope and such. You are generally shielded from bugs through normal means.
This is, of course, a major point for open source where everyone gets bugs serviced, not only those that pay a bounty for them. Basically closed sourced software does carry some huge risks unless you have big money and leverage to put on their development teams. Open source, not vendor contracts, are the real protection for SMBs here. If this is seen as a real concern, I think that broader thinking provides more risk mitigation than just handing more financial resources over to a vendor that witholds fixes and intentionally leaves bugs in their products to encorage people to need to pay for them.
-
@scottalanmiller said in The argument for official support vs third party support:
I have to agree, if the premise is that people keep choosing software that doesn't work under many conditions and the vendor doesn't fix when a bug is found... sounds like software to generally avoid.
That is my stance. If the software is so buggy, that only the developer would fix the issue, when enough customers complain then I'd generally just avoid the software. Same with any software. If it's broken, why would I use it. I'd find a better option.
-
Even when I pay for support, I expect that to be support, not "paying to make the product work as advertised." If it doesn't do that, I shouldn't have had to have paid for it in the first place.
-
It's an important discussion. SMBs often fall into a space of being either overly tied to vendor support or completely eschewing it. Neither of which really makes sense. There is a sensible place and time to have vendor support, and loads of sensible places not to have it.
-
I think it's a critically important conversation to have. Support is support, it's primary benefit shouldn't be to fix bugs in the software you paid for.
It is there to make sure that if your system goes belly up, that you have the support you need to be rectified and functional within the time frame that you planned for.
-
@scottalanmiller said in The argument for official support vs third party support:
I have to agree, if the premise is that people keep choosing software that doesn't work under many conditions and the vendor doesn't fix when a bug is found... sounds like software to generally avoid.
This isn't unique to Hypervisors, this is an issue on switches, routers, and other devices that run proprietary binaries. We had at the MSP tons of bugs we got fixed by Brocade, HPE and Cisco (I'm convinced the RSTP standards committee meetings required everyone drink a bottle of vodka before they got started).
One trend I've noticed is a lot of bugs are now ecosystem bugs. Issues with API's called by backup vendors (who may be doing stupid things that they need to fix). Hardware driver and firmware bugs (Intel basically ignored massive problems with multicast retransmits and LSO/TSO on their 540 NIC for the better part of 4 years). HPE's Switch to Adaptec controllers on Gen 10 came with some.... interesting stability issues. Storage bugs have become more pronounced as people push the limits of SATA SSD's (SATA tunneling protocol was a bad idea in general) and technologies that may have been stable at one time (SAS buffering) just scaled poorly. Your OS vendor having an HCL program (RedHat, VMware do this, Microsoft's 5 pack of Heineken and running a script requirement tends to be a bit weaker) that mandates the other engineers will work with them to get fixes on things. Now you could argue this is the hardware vendors problem (it is) but many times it's the OS/Hypervisor who ends up being the one who forces the issue or puts resources into the RCA. Given the massive monoculture for NIC's, HBA's, and drive controllers this is becoming more "fun". In some ways I"m hopeful things will tame down a bit (the death of SATA and SCSI will help) but in other ways, I'm a bit worried (RCoE could be a mess).
My experience came from working with hundreds of customers, some of whom push the extremes of things, but also SMB's who just lost the hardware/driver/firmware lottery, or discovered (especially with XenApp) that they were the first people to try to use a feature (vGPU offload used to be fun!).
While it's true most vendors will eventually fix critical bugs, if you're not a paying customer they still might take a few weeks. A few weeks of crashes to a company generally isn't cool and is a great way as an MSP to get fired.
The other interesting side of vendor support is the new classes of proactive support (Things like Infosite by Nimble, and Skyline and Humbug at Vmware) where you have telemetry systems written into the software that phone home performance, logs and config data and allow for machine learning systems on the vendor to "Predict" issues across multiple customers. MSP's can't aggregate 500K customers to identify corner cases like this. Cloud providers can, but it's interesting to see traditional infrastructure companies adopt the same model of correlation and continuous improvement.
If you're going to complain about support practices of commercial companies I'd argue the one that bothers me the most is vendors that hide their KB systems and admin guides from people who are not paying. WHAT do they have to hide!
-
@dustinb3403 said in The argument for official support vs third party support:
I think it's a critically important conversation to have. Support is support, it's primary benefit shouldn't be to fix bugs in the software you paid for.
It is there to make sure that if your system goes belly up, that you have the support you need to be rectified and functional within the time frame that you planned for.I'd argue where we are going should be for people to predict that an environment is going to go boom and fix it beforehand. MSP's tried to prevent this by doing various things (we did SNMP monitoring, WMI, log analytics and pattern matching for previous issues we had found), but the scale that some of these guys can do. https://www.nimblestorage.com/technology-products/infosight/ 86% fixing problem before impact is pretty damn impressive, as well as the ability to RCA things that are not exactly "their bag/fault" is where stuff is going. Just pointing a finger at the network isn't a solution. Mapping what's wrong (and then assisting with fixing it) is the future. Vendors are starting to bleed into the MSP's capabilities here (and do it better given the size of the data they can collect).
-
@scottalanmiller said in The argument for official support vs third party support:
Microsoft doesn't even offer support in the SMB
They offer pay per ticket support. It's questionable the quality, but it does exist and if you find a bug they give you your money back (We rarely paid for tickets, but sadly it took weeks to get resolution sometimes).
The issue of paid vendor support isn't that a bug that's critical will not get fixed, it's just it may not get fixed in the timetable you need. Paying for expediency shouldn't be that shocking. We had a Brocade switch that we found a nasty RSTP bug in. We had a hotfix the same day. I found similar bugs in a Linksys switch. I never saw a resolution on it (They just didn't care). A Red Escalation (Priority 1 bug) from a paying customer will trigger engineers to work weekends and get overtime pay. A report from someone in a forum having an issue will get a "send me the logs and I'll look at it when I can" from most vendors.
-
@storageninja said in The argument for official support vs third party support:
@scottalanmiller said in The argument for official support vs third party support:
Microsoft doesn't even offer support in the SMB
They offer pay per ticket support. It's questionable the quality, but it does exist and if you find a bug they give you your money back (We rarely paid for tickets, but sadly it took weeks to get resolution sometimes).
The issue of paid vendor support isn't that a bug that's critical will not get fixed, it's just it may not get fixed in the timetable you need. Paying for expediency shouldn't be that shocking. We had a Brocade switch that we found a nasty RSTP bug in. We had a hotfix the same day. I found similar bugs in a Linksys switch. I never saw a resolution on it (They just didn't care). A Red Escalation (Priority 1 bug) from a paying customer will trigger engineers to work weekends and get overtime pay. A report from someone in a forum having an issue will get a "send me the logs and I'll look at it when I can" from most vendors.
That was my point, they offer it AFTER you find a bug and they can decline it AFTER you are stuck with them.
-
@scottalanmiller said in The argument for official support vs third party support:
That was my point, they offer it AFTER you find a bug and they can decline it AFTER you are stuck with them
Not all companies subscribe to the first rule of acquisition. Gint was wise, but software vendors tend to not be that bad. I have seen this happen, but it tends to be with vendors who are more concerned about their next customer than their existing ones for revenue. Your large enterprise vendors (Redhat, Juniper, HDS, Oracle, VMware) tend to not do this. Startups and hardware vendors tend to be the worst offenders (Hardware because it can cost them a lot for a recall like the Intel Clock issue that impacted the Atoms), and startups because they may realize that their first 100 customers are not going to be the kind of customers or industry that gets them to 10K customers and they will "Pivot" and abandon a field.
The reality is depending on the platform the labor to migrate (or to account for missing capabilities) isn't free. I've seen people replatform their monitoring system every 3 months. This just isn't a good idea as the value of the monitoring solution tends to come from the time invested in customization and training etc.
What's your reaction if you hit a buggy driver from Intel. Buy a different NIC, or call the vendor to see if you can get a fix or a workaround for it? That I guess fundamentally determines if you're the kind of person who buys vendors support.
-
@storageninja said in The argument for official support vs third party support:
What's your reaction if you hit a buggy driver from Intel. Buy a different NIC, or call the vendor to see if you can get a fix or a workaround for it? That I guess fundamentally determines if you're the kind of person who buys vendors support.
This is dependent on the cost of the system that is reliant on this NIC. If it possible to get a different model NIC for $1000 that will work, but it would cost you 3 days of down time to get the part-in and installed.
Is it worth it?
Can the vendor supply you a patched driver before then? Is the vendor charging you to create a custom driver? Can the vendor even get a working driver to you faster than you can get the part?
-
@storageninja said in The argument for official support vs third party support:
What's your reaction if you hit a buggy driver from Intel. Buy a different NIC, or call the vendor to see if you can get a fix or a workaround for it? That I guess fundamentally determines if you're the kind of person who buys vendors support.
Commodity parts, get one that works and move on. Paying for unique support and code changes to support a single SMB use case rarely works. As Dustin points out, I can swap the part in an hour. I can't get support to understand what the issue is in that amount of time. Vendor support, in that example, represents high cost and high risk. I know that I can swap parts and that I can swap them now. I don't know if the vendor can fix the issue or if they will in any reasonable amount of time.
-
@scottalanmiller It is the value of time spent troubleshooting vs time spent replacing. Sometimes it makes more sense to replace something than try and figure out what it is. If I spend 2 hours trying to figure something out on a $100 printer, I should have just replaced it.
-
@wrx7m said in The argument for official support vs third party support:
@scottalanmiller It is the value of time spent troubleshooting vs time spent replacing. Sometimes it makes more sense to replace something than try and figure out what it is. If I spend 2 hours trying to figure something out on a $100 printer, I should have just replaced it.
Yup, which is often the case with operating systems, too!
-
@scottalanmiller said in The argument for official support vs third party support:
@wrx7m said in The argument for official support vs third party support:
@scottalanmiller It is the value of time spent troubleshooting vs time spent replacing. Sometimes it makes more sense to replace something than try and figure out what it is. If I spend 2 hours trying to figure something out on a $100 printer, I should have just replaced it.
Yup, which is often the case with operating systems, too!
Exactly. A re-image solves almost everything.
-
@wrx7m said in The argument for official support vs third party support:
@scottalanmiller said in The argument for official support vs third party support:
@wrx7m said in The argument for official support vs third party support:
@scottalanmiller It is the value of time spent troubleshooting vs time spent replacing. Sometimes it makes more sense to replace something than try and figure out what it is. If I spend 2 hours trying to figure something out on a $100 printer, I should have just replaced it.
Yup, which is often the case with operating systems, too!
Exactly. A re-image solves almost everything.
Yup, the DevOps philosphy. Rebuilt to know good rather than trying to troubleshoot the unknown.
-
Obviously, there is value to knowing why something occurred. For instance, when it occurs repeatedly. Then you have to identify the root cause.
