AV-Comparatives.org, bullying, censorship and financial deals-continued...

[Guest]

[IGLSecurity.Brad]

How do you believe it should be validated? It is really possible that a AV testing group can be validated?

[Melih]

How do you believe it should be validated? It is really possible that a AV testing group can be validated?

of course!

Auditing exist and for well documented processes, there are many firms out there who can audit processes and provide their report. Its not about Auditor testing malware...its about auditor auditing the tester while the tester is testing malware...it ain't rocket science and many respectable firms out there who will do it.

[IGLSecurity.Brad]

Then what about other AV/Firewall testing groups? Have the other groups of testers that have tested Comodo (matousec, VB, AV-Test| I can't remember which of those last 2 Comodo was in) been validated?

[Melih]

Then what about other AV/Firewall testing groups? Have the other groups of testers that have tested Comodo (matousec, VB, AV-Test| I can't remember which of those last 2 Comodo was in) been validated?

No. But they don't call themselves "independent" or claim that only good AVs make it although they can't test all AVs, or hide financial details. Also likes of Matousec makes all their tests available so that you can reproduce their tests, unlike AV-Comparatives.

AMTSO would have been a great initiative to standardise all this but AV-C dropped out of it, most likely because they didn't want External Auditors to audit them.

[Melih]

http://www.matousec.com/matousec/about-us.php

http://www.matousec.com/services.php

Here they are very descriptive about what they do, and they clearly explain their services and also mention that they charge for them.

[spainach_12]

Yes, they DO get paid by AV vendors and yes they MIGHT be influenced by money to put some AV products higher in rank than others. However this cannot be proven and should remain a speculation.

Well said, and as speculations, should not be spread as truths without well-grounded proofs (that is to say, proofs that would not open to a wide range of possibilities). If one speculates, a rigorous research of proofs must be done and then properly documented. This would make for a more persuasive argument and would thus be legitimate in its claims. This mudslinging acts are further marring the situation. It's blown way out of proportion in my view.

The FACT here is that AVC has a BIG influence on a products marketing and sales if AVC ranks it's products DETECTION on high. The flaw here is that DETECTION means NOTHING compared to PROTECTION. They have an outdated view of AV testing by believing that DETECTION is THE SAME AS PROTECTION. If you detect malware, then you can stop it, right? Wrong! There are many AV's that detect malware AFTER the damage has been done. You might have a product that detects 99% of malware, but what good is it if they are not stopped? And what about the 1%? That is still 1.000.000 pieces of malware let through out of 100.000.000.

"Old as they may be, I still find them rather relevant as they do show you the capacity of AV's in case of emergencies. Prevention is indeed a better option, but it is not expected that every malware can be prevented. This is still as serious as it can be because if every other av company focused on prevention, and it so happens that by some misfortune a prodigious cracker manages to slip a virus inside computers, then what of the capacities of the av's to remedy such things? What would become of the users?"

see here: https://forums.comodo.com/other-security-products/retrospective-test-november-2011-t78699.0.html

As was stated in the Fee agreement, vendors can use the AVC logo for marketing purposes, meaning that AV vendors can put an AVC logo on their products, showing that it was tested by AVC and thus you can 'trust' it, boosting the products 'trust' and feeling of 'safetey', while in reality the product may provide less protection that those that scored very very low on avc (i.e: Comodo?).

Well, I don't view it as that, that detection should be the sole basis for trusting a product. I understand that systems are just as unique as its users (as a matter of fact, I'm preparing to write an essay about this. still doing some more research), hence, needs vary. Detection is ONE of the MANY criteria to consider in choosing a product. This is something not many people understand.

"Malware detection rate is still one of the most important and reliable factors in determining the effectiveness of an anti-virus engine which works without asking for user interaction, decision or opinion." -From AV-C

This is a clear attempt in objectifying the basis for choosing products. Because detection can be objectively measured, but not protection (because protection is subject to vary per person depending on the current level of knowledge he possess about his system. Objectifying protection cannot be successfully done without causing misunderstanding. For example, product A claims to protect 100%, but has compatibility issues with system A and causes system A to crash, or product B claims to protect 100% but is entirely user-dependent and since user B has little working knowledge of systems, allows malware in anyway. Protection cannot be measured as it is a very general criteria and varies from person to person. One way of trying to measure it is by measuring INDIVIDUAL COMPONENTS of protection i.e. Detection rates).

For me AVC is like the FDA, creating false trust. They claim they use an AUTOMATED testing procedure and release results every 3 months or so? How many products do they test, about 20? How much does it cost to hire 1 person to test 20 AV products in 3 months time manually (like a real user would, i.e. Languy99 on YouTube) ? AV products will be used by people and NOT by robots! Or are they using this 'automated testing' thing as a protection when people claim that the tests are not accurate to reply that the tests were performed by an automated system and thus is more accurate than a human tester?

Their methodology is included in their website and can be downloaded from there. I have attached the file here for faster access.

They include in their test summaries these lines:
"Do not take the results as an absolute assessment of quality - they just give an idea of who detected more, and who less, in this specific test... Readers should look at the results and build an opinion based on their needs. "

"We do not give any guarantee of the correctness, completeness, or suitability for a specific purpose of any of the information/content provided at any given time."

It would seem that only the vendors propagate this "Detection is Everything!" marketing strategy.

Bottom line,my opinion is that AVC is just a "marketing through trust" merchant where you can buy "trust" by having your product tested and results put in a 'roulette' of other vendor's results, hoping your product would come out as highest ranking and use this in your marketing. For a couple of thousand dollars, you can also enter your AV into this roulette...who knows, you might 'win' this time?

Trust cannot be commodified as it is an abstract object and without a clear definition of its nature. Trust cannot be forced upon anyone. Only influenced. People choose the highest detection? Ok, sure. What about others? My colleagues, I and my brother choose products that are consistent in their records. Which means we shift through records to identify reliable companies. Since there has been no formal statistical collection of the number of users and how they base their judgment on choosing products, we cannot assume that AV-C has such a huge influence. Maybe on the more technically inclined, but certainly not average users. In my university, only three of ten people (on a sample size of 1000 users collected in ten months time. This is an informal statistical collection by the way done in 2010 by  me and two other friends in their colleges) are aware of AV-C and only 1 of ten base on AV-C results as a SOLE criteria. So claiming that AV-C has a huge impact on the general public does not seem to appeal to me much.

[spainach_12]

Melih, I'm very well aware of your thoughts concerning protection vs detection/cleaning. You even have posts about it. And I applaud you for them.

However...

what exactly do you wish to happen with this argument of yours between av-comparatives? Are you threatened by other AV companies as well? As I see it, AV-C have done nothing as of now that would gravely affect your share of the market. You do realize of course that by your claims with AV-C, you are also stating that other vendors are using underhanded tactics, and thus publicly discredit them. And you are discrediting companies such as Norton, GData, Qihoo, Avira, Kaspersky, Eset, FSecure, Bitdefender and basically every other vendor that scored high now and in the past, implicitly (or perhaps may be even explicitly) stating that they are paying AV-C more than what is required to have better results.

Would it not be better if say you have other views for testing antimalware products to establish your criterias of how testing should be done? Then have this critiqued by various sources and then post it as research material. I'm sure there are ten or a few others more who are more than willing to collaborate with you.

I see no reason why you should engage in such a futile argument as this with AV-C.

[spainach_12]

And for those others who seem to be in paranoid mode,

I am in no way affiliated to any company (for crying out loud people, this isn't one of those conspiracy movies we see in the theaters). I am not a spy. I am not anything but a Literature major whose hobby is internet security. Speculate if you must, but back it up with legitimate arguments and not with proofs that you just happened upon while searching for keywords. I myself don't know the truth. So I seek it.

Calm down. We have yet to see the truth so don't go jumping into conclusions and spreading word about it the first chance you get. Otherwise, we'll see this thing blown too much out of proportion and soon lawsuits and mudslinging and other forms of politicking follow. This is tiring me.

[Melih]

Melih, I'm very well aware of your thoughts concerning protection vs detection/cleaning. You even have posts about it. And I applaud you for them.

However...

what exactly do you wish to happen with this argument of yours between av-comparatives? Are you threatened by other AV companies as well? As I see it, AV-C have done nothing as of now that would gravely affect your share of the market. You do realize of course that by your claims with AV-C, you are also stating that other vendors are using underhanded tactics, and thus publicly discredit them. And you are discrediting companies such as Norton, GData, Qihoo, Avira, Kaspersky, Eset, FSecure, Bitdefender and basically every other vendor that scored high now and in the past, implicitly (or perhaps may be even explicitly) stating that they are paying AV-C more than what is required to have better results.

Would it not be better if say you have other views for testing antimalware products to establish your criterias of how testing should be done? Then have this critiqued by various sources and then post it as research material. I'm sure there are ten or a few others more who are more than willing to collaborate with you.

I see no reason why you should engage in such a futile argument as this with AV-C.

Have you read my blog to see what started this?

[w-e-v]

what exactly do you wish to happen with this argument of yours between av-comparatives? Are you threatened by other AV companies as well? As I see it, AV-C have done nothing as of now that would gravely affect your share of the market. You do realize of course that by your claims with AV-C, you are also stating that other vendors are using underhanded tactics, and thus publicly discredit them. And you are discrediting companies such as Norton, GData, Qihoo, Avira, Kaspersky, Eset, FSecure, Bitdefender and basically every other vendor that scored high now and in the past, implicitly (or perhaps may be even explicitly) stating that they are paying AV-C more than what is required to have better results.

I see no reason why you should engage in such a futile argument as this with AV-C.

Are you serious? (I am not Melih, but I was urged to answer this, sorry for being nosy)

Have you read everything he posted? Then you would understand why.

Internet is about sharing information in many ways.
This information heis sharing is for the benefit of all of us, and believe it or not, that includes AV-C itself.

[Melih]

who agrees with this statement from AV-comparatives?

"Malware detection rate is still one of the most important and reliable factors in determining the effectiveness of an anti-virus engine which works without asking for user interaction, decision or opinion."

[Solarlynx]

who agrees with this statement from AV-comparatives?

"Malware detection rate is still one of the most important and reliable factors in determining the effectiveness of an anti-virus engine which works without asking for user interaction, decision or opinion."

I guess they just maintain the simplest way to test avs - by detecion of 'dead' malware - which is inactive and the only thing is tested that av recognizes an infected file.

[The CEO]

who agrees with this statement from AV-comparatives?

"Malware detection rate is still one of the most important and reliable factors in determining the effectiveness of an anti-virus engine which works without asking for user interaction, decision or opinion."

I do. It's a "motherhood' statement. Any questionabilty of its proposition is negated by it's own qualifying condition. And if I may say so, whether intentional or not, a statement that neatly draws a distinction between PC users who expect (and are even prepared to pay money for) their security software to make the majority of decisions on their behalf, and those who prefer to be informed by their software but take responsibilty themselves.

[clockwork]

who agrees with this statement from AV-comparatives?

"Malware detection rate is still one of the most important and reliable factors in determining the effectiveness of an anti-virus engine which works without asking for user interaction, decision or opinion."

I agree to that. If i choose for someone else an antivirus engine product that has to run "without asking for user interaction, decision or opinion", then it HAS TO have a detection rate which covers the usual day malware, and the heuristics.
If you choose an "autonom" product, this product should be able to act like that.

This avC statement has been made about "anti virus engines". And for an antivirus engine, this statement is right. I dont read hips or sandboxes.

My first antivirus product was a diskette for the amiga. There havent been updates. I just had this diskette. And if back then there would have been malware tests, the detection rate result had shown the effectiveness of that antivirus. Its logical.

You can argument about, if antivirus is enough. But you can not negate the tests which test antivirus. That doesnt make sense.
If it would make sense, comodo could have a detection rate of 1%, and you could say: Let this program run on its own, and you will be protected. That is wrong. I saw peoples machines getting infected after a few days, even though, they had "a host intrusion protection".
The antivirus is able to safe the user if he makes bad decisions, or if he cant make decisions on his own. But it needs a detection rate to achieve that. This is tested. This IS a factor for valuating the use of an antivirus engine product.

[spainach_12]

Are you serious? (I am not Melih, but I was urged to answer this, sorry for being nosy)

Have you read everything he posted? Then you would understand why.

Internet is about sharing information in many ways.
This information heis sharing is for the benefit of all of us, and believe it or not, that includes AV-C itself.

This is not yet information, but speculation.

"Here is my opinion about The issues with  AV-Comparitives.org and why they try hard to “censor” anyone who reveals this:"

http://www.melih.com/2011/11/27/av-comparatives-org-bullying-censorship-and-financial-deals-with-anti-virus-vendors/

Have you read my blog to see what started this?

Yes, I have. You've pointed me to this beforehand, remember? It would be rude of me not to read it. But like I said, it's better to put it in research than go this way. Otherwise, it's all speculation (which are  defamatory).

Many have done the same and placed it in research. This is information. Well grounded, tested and proven. If they make any speculation they attempt to prove it through research and objective investigation.

http://www.icgg.org/

That is one example.

What is thought to be otherwise unquantifiable is in fact measurable and researchable. This is my last post. it's still up to you anyway.

[Tags:]