Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?
-
@Richard_Cylance said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
This article clearly demonstrates the need for the industry to update their testing practices. Testing needs to change, and we've been saying it for years. We will continue to call out bad testing practices. Matt and I have said it before on here- don't trust us. Download your own samples, run it against your AV solutions and our offering. There is a need to test against files never seen before, using the same methods that malicious distributors are using.
But if we do this, aren't we going to get threatened by the lawyers? This is exactly what Ars Technica claims people are not allowed to do. Just because you tell us to do it doesn't mean we are allowed to discuss it. This is what we are asking... did Ars Technica lie, or is "do your own testing" not actually something we are allowed to share (read: if we aren't free to discuss it, we are into Nutanix territory.)
-
We are asking a direct question, just need a direct answer.
Did Ars Technica report something false? Or did Cylance actually do what they said?
-
-
@scottalanmiller Look at it as an interpretation of the situation. For the longest time we have been fighting against the accepted mindset of security. Now, no matter where you stand on what AV to use, there is no denying that advancement hasn't been the strongest point of the industry. Cylance changed that with a new way of thinking, and when the OGs of the company started out on the road, they fought back this wall.
You are more than welcome to test our product out, use malware you find, and post your results. Assuming you are talking about the Sophos/Cylance issue, there were higher up issues with the reseller than I am privy to, so I won't speak to that. But the test they showed was flawed, and we called them out for it. That is the past though.
Every test I have ever done in front of an audience or a customer I have invited attendees to check my settings. I update all the products I test against, enable all the features I have that are viable for the test (don't need anti-phishing layers on for a malware execution test for example), and ensure all signatures/dats are downloaded and updated, as well as full connection to their servers is there. I don't do that for Cylance - I run a version on my test machine from September 2015 and often run it disconnected from the web. Again, I ask any attendee to come up and verify, recommend changes if they feel the need, and run the test for them.
But in the end, I am a vendor, and we want people to trust the results themselves. You wouldn't buy a car based off someone's presentation alone; you're going to test drive the metal machine. Same thing here; don't trust me or any other vendor.
-
@Richard_Cylance said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
@RojoLoco If you approach me in a way that clearly gives me no room to present our side, then no matter how we work in this, you will always have the wall.
Look - I said it above - SW 15 was a poor representation and action on our end. We own that. Matt and I joined up and have been working with our experiences and skills to tell more and more of the story, and it's been incredible growth in the right direction for our company.
I posted this on SW-
" I invite attendees of all my shows to come up and verify my settings on all the competitor setup. I'd be happy to have a one-on-one with you and let you have full control and run the demo on my machine (so you don't get infected).This article clearly demonstrates the need for the industry to update their testing practices. Testing needs to change, and we've been saying it for years. We will continue to call out bad testing practices. Matt and I have said it before on here- don't trust us. Download your own samples, run it against your AV solutions and our offering. There is a need to test against files never seen before, using the same methods that malicious distributors are using.
Independent tests that don't use real-world methodologies are ineffective. AV-Test and NSS Labs are endeavoring to make tests more akin to the real world, and we support them in their efforts. While no testing house is perfect, they are making changes that serve the world at large."
I feel like this all just highlights the need for fair testing practices, standardized by testing agencies, not specified by vendors. I'd love to hear your side, but understand that there is already damning evidence against Cylance, so I'd need some pretty convincing materials that came from a neutral source, not from Cylance.
And the removal of your avatar is... well..... telling? suspicious? not sure what word I'm looking for.
-
@Richard_Cylance said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
You are more than welcome to test our product out, use malware you find, and post your results. Assuming you are talking about the Sophos/Cylance issue, there were higher up issues with the reseller than I am privy to, so I won't speak to that. But the test they showed was flawed, and we called them out for it. That is the past though.
What do you mean it is the past? It's a news report today. Instead of stating that the tests were flawed, it became a legal matter - a legal matter that puts Cylance squarely on the opposite side of the court as their customers.
This is not past, until Cylance posts a retraction and removes the legal blocks, it is current, it is right now.
-
@Richard_Cylance said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
But in the end, I am a vendor, and we want people to trust the results themselves. You wouldn't buy a car based off someone's presentation alone; you're going to test drive the metal machine. Same thing here; don't trust me or any other vendor.
Individuals can't reliably run good testing. Anything that requires this is the same as saying we shouldn't consider the product. We want industry testing that is standard, and that requires no legal blocks on testing.
-
@RojoLoco said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
I feel like this all just highlights the need for fair testing practices, standardized by testing agencies, not specified by vendors.
Yes, very much so. Independent third parties without constraints need to have public testing methods and public test results. Always difficult as nearly all testing agencies are sponsored by one vendor or another.
-
@Reid-Cooper said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
@RojoLoco said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
I feel like this all just highlights the need for fair testing practices, standardized by testing agencies, not specified by vendors.
Yes, very much so. Independent third parties without constraints need to have public testing methods and public test results. Always difficult as nearly all testing agencies are sponsored by one vendor or another.
Something like SANS or similar to that nature?
-
@scottalanmiller "One vendor, CrowdStrike, even pulled out of the NSS Labs tests and revoked the testers' license, then attempted to obtain a restraining order to block publication of the results NSS had obtained. "CrowdStrike filed suit in US Federal District Court against NSS Labs to hold it accountable for unlawfully accessing our software, breaching our contract, pirating our software, and improper security testing," a company spokesperson wrote in a post to CrowdStrike's blog. "Regardless of test results (which we have not seen), CrowdStrike is making a stand against what we believe to be unlawful conduct." The court denied CrowdStrike's initial request for a restraining order, but the case has yet to be decided.'
We aren't CrowdStrike.
-
@Reid-Cooper The problem there is funding. As the article talked about, a lot of the testing labs are funded through subscription.
If you can find a way to build a testing lab that tests 100% un-funded by the industry it's testing, I can promise you we would be the first interested party to submit and participate.
-
@Richard_Cylance said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
@Reid-Cooper The problem there is funding. As the article talked about, a lot of the testing labs are funded through subscription.
If you can find a way to build a testing lab that tests 100% un-funded by the industry it's testing, I can promise you we would be the first interested party to submit and participate.
A broad problem that we have is a need for general testing in IT. Labs for everything. No one has labs for any purpose, storage, AV, operating systems, nothing. No one has a financial interest in funding labs except for the businesses that use IT at the end of the day, and that doesn't work as the freeloaders break the system.
-
@scottalanmiller Kickstarter.com
#makelabsindependentforonce
-
@Richard_Cylance said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
@scottalanmiller "One vendor, CrowdStrike, even pulled out of the NSS Labs tests and revoked the testers' license, then attempted to obtain a restraining order to block publication of the results NSS had obtained. "CrowdStrike filed suit in US Federal District Court against NSS Labs to hold it accountable for unlawfully accessing our software, breaching our contract, pirating our software, and improper security testing," a company spokesperson wrote in a post to CrowdStrike's blog. "Regardless of test results (which we have not seen), CrowdStrike is making a stand against what we believe to be unlawful conduct." The court denied CrowdStrike's initial request for a restraining order, but the case has yet to be decided.'
We aren't CrowdStrike.
Care to comment on the rest of the article instead of only the little bit that doesn't concern Cylance?
Hint: Sophos obtained a copy of Cylance Protect from a reseller in order to conduct its own test, then posted the results in a YouTube video. Cylance then "contacted the reseller who provided access to the Cylance PROTECT product, citing license compliance concerns and threatening 'retribution' if the reseller involved did not demand that Sophos withdraw the video immediately," Schiappa wrote. "This left the reseller in fear of a lawsuit." Sophos pulled the video to protect the reseller. But, Schiappa claims, Cylance has continued to do public demos using Sophos products in violation of their licensing terms—and after renewing the license through a Sophos reseller. Cylance says that the company has since stopped using Sophos' software in its "Unbelievable" demos. And Gale rejected Schiappa's assertion that the demos were unfair.
I think we all understand about protecting the licensing terms, what we can't accept is the removal of a comparative analysis with no explanation. Also, yes, I realize Sophos is a competitor, that makes things even worse as they were attempting to utilize what they thought was a proper license.
-
@Richard_Cylance said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
@scottalanmiller Kickstarter.com
#makelabsindependentforonce
Hmmmm.... Kickstarter....
-
@travisdh1 We commented on that part last year....when it happened. https://www.cylance.com/cylanceprotect-vs-smoke-and-mirrors
I've been commenting on everything else throughout the thread.
-
@Richard_Cylance said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
@travisdh1 We commented on it last year....when it happened. https://www.cylance.com/cylanceprotect-vs-smoke-and-mirrors
And that post is not a bunch of smoke and mirrors? All you did was turn the accusation around on your accusers. Well, that and insist that your testing methods are revolutionary and paradigm shifting...
-
@Richard_Cylance said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
@travisdh1 We commented on that part last year....when it happened. https://www.cylance.com/cylanceprotect-vs-smoke-and-mirrors
I've been commenting on everything else throughout the thread.
I'll attempt to make this easier to understand by asking a different way.
What confidence do customers have that Cylance won't threaten legal action when attempting to talk about the product publicly?
-
@travisdh1 said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
@Richard_Cylance said in Cylance Unbelievable Tour Lives Up to Name, Can Cylance Be Trusted?:
@travisdh1 We commented on that part last year....when it happened. https://www.cylance.com/cylanceprotect-vs-smoke-and-mirrors
I've been commenting on everything else throughout the thread.
I'll attempt to make this easier to understand by asking a different way.
What confidence do customers have that Cylance won't threaten legal action when attempting to talk about the product publicly?
No need to be pandering or rude. I get what you are asking.
You're trying to cross two different situations. The issue we had was not with a customer, but with a reseller breaking their terms and conditions they agreed upon. Sophos' video showed the settings they used, and we called them out on it. They changed their T&Cs to stop public testing, we removed them from our list of products we use in public testing. Then we all moved on.
You can talk about the product all you want. Share the results of your POC, use, or results. That's all on you and your experience, and there are plenty of threads all over the web that have those kinds of conversations.
-
@Richard_Cylance The issue is that you're fighting a problem of credibility. I read the Ars article a couple days ago and read through this thread and your responses to this tread's participants and while it feels like the folks here are asking some pretty direct questions, they are getting marketing speak in return. Sometimes, especially when credibility has taken such a hit, whether the stories are true or not, you have an uphill battle to climb and skirting questions with round-about answers won't get you where you need to be.
Having said this, I agree that a standardized approach to testing would be valuable. Problem is, malware is hardly "standard" with different variants behaving differently from one day to the next as these evolve. So as much as I'm with you on "let's get an open, public, standard way of measuring effectiveness", I don't think that's very realistic and I certainly don't believe this will address Cylance's credibility. The same would be true of any vendor in the same position.
My recommendation...stop...put away your marketing hat, and deal with questions head on. Failure to do so only casts further shadow and doubt.
