CAV and CIS detection rate

I have used CFW since version 2 and believe it to be one of the best firewalls. I have never used CAV because despite improving reviews in my opinion it still lags behind the best antivirus programs.

There have been many requests on this forum for CAV to be tested by AV-comparatives (as THE definitive AV comparison) but this doesn’t seem likely to happen in the near future given the following statement below from the recently posted Security Survey 2011 on the AV-comparatives web site:-

“Unfortunately, although Comodo is a high demanded test candidate in this survey, Comodo did not apply to participate in our 2011 test series, but they agreed to be tested separately in a Single Product (On-Demand) Test.”

I wait with interest to see the results of the Single Product (On-Demand) Test, but unfortunately although this type of test is useful it doesn’t provide direct comparisons with other IS suites.

It seems that some other people share the view that CAV detection rates are below other top AVs, but they go on to state that it doesn’t matter because the “Default Deny” and “Automatic Sandbox” technology in CIS will catch almost all malware, or that CAV is “only there for usability” (can someone explain what that means?).

I would appreciate some feedback on some questions and observations:-

  1. Why hasn’t Comodo applied to participate in the AV-comparatives 2011 test series?

  2. If CIS “Default Deny” and “Automatic Sandbox” technology is the ultimate in stopping zero-day malware, why isn’t CIS consistently at the top of recent IS suite comparison tests using large samples of malware?

  3. I would suggest that replacing CAV with any AV that has superior detection and using it with CFW would provide better overall protection than CIS by potentially stopping more malware before it got to D+ or the Sandbox thus reducing the number of “Default Deny” alerts (which might be responded to incorrectly by the user) and reducing the number of programs being run in the Sandbox (because some programs don’t work in a Sandbox). It would also potentially stop malware missed by CIS because CFW like all anti-malware cannot give 100% protection no matter how it is configured or what technology it uses.

Do you think the AV comparitives test can test how well CIS can protect you?

or…

Is it a test of: Putting all the static malware to a folder and running the AV to see what it can detect?

have you seen this? https://forums.comodo.com/news-announcements-feedback-cis/cis-certifications-test-results-reviews-t61263.0.html;msg431091#msg431091

and you have read this? https://forums.comodo.com/news-announcements-feedback-cis/can-you-measure-the-speed-of-an-airplane-with-a-car-speedometer-t68368.0.html;msg484175#msg484175

I would like to see both. Pure CAV detection test i.e CAV’s signatures, heur & may be cloud. And also would like to see Protection test i.e CIS full suite test. Both the tests with default settings coz majority of users use this setting. Pure detection test will show where CAV stands against the good & famous standalone AV’s. Full suite test will show its protection strength as a whole.

As an average user this is just my opinion so plzz dont give me here the stuff like CAV is just for usability, CIS is a protection & not detection thing, etc which I have read here in many threads 1000’s of times.

Its simple if you have a detection mechanism in your security suite people want to know how good it is & where it stands against others. Even if it is there for usability, users wants to know in usability state too how much effective it is compared to others.

Thanxx
Naren

how many malware did you have in your pc in your everday use that affected your computing while using Comodo? (you will see that number is almost nothing compared to other vendors and that is whats important)

Obsession with old testing methods for a new product :slight_smile: . You can’t test the speed of an airplane using a car speedometer.

so lets say the number is 91%…then what? Comodo wil still be more user friendly. how many times the few malware we might miss (and others miss too btw) will cause an issue (they are all sandboxed) vs if the user was using other av that didn’t sandbox automaticaly all unknown and let what they didn’t detect in.

Comodo is a MUCH more usable product even if it ZERO detection ratio!

AV products miss literally 80%+ of all new malware until they get a sample of it and build signatures/heuristic/rules.

So here is your option: Be wide open to all new malware vs be protected using auto sandboxing.

So answer me this Naren: Which product will be more usable against all the unknown malware? (btw: i hope you understand that all new malware in majority goes undetected and these detection tests you refer to are in majority for known malware), a traditional AV that will not recognise the new malware and allow infection or Comodo that will sandbox and stop infection?

which is more usable and offer protection?

Melih

Melih,

the problem is that there are still numerous number of malware files that are recognized as safe when scanned online.
Comodo lab receives submissions from users every day.
This should be solved ASAP because “Default Deny” approach won’t work properly.

BUT whats the harm getting the tests done. Ok I agree the tests are for known malwares. So let us see howz the CAV detection against known malwares, what average percentage of usability it provides by detecting known malwares, how it compares with others against known malwares??

Thanxx
Naren

The standard AV-comparatives test would compare CAV detection rate and false positives to those of other AVs. I am well aware that this does NOT reflect the total protection ability of CIS because CAV by itself doesn’t include the technology in CFW/D+/Sandbox for handling unknown and suspicious threats (yes I have read the posts on the car speedo and airplane). You seem to imply that this test only measures static malware detection which may be true for CAV but not necessarily for other AVs (most AVs use more than just signatures and heuristics). I still maintain that replacing CAV with another AV with a better detection rate while using the same CFW/D+/Sandbox configuration would provide better protection than CIS. I believe that AV-comparatives is the best way to compare AV detection rates.

The Single-Product test will give an indication of whether CIS is good, bad or indifferent, but it won’t directly compare CIS with other IS suites. I have also read most of the posts on CIS tests and like most well-known anti-malware products it does well in some and not so well in others.

so what you are saying is: don’t use the blacklisting we provide…use another one instead?

so what will happen to a malware that the other AV doesn’t detect?

The D+ will show a warning…then the user will complain that we are showing an alert :slight_smile:

You will be better protected with full CIS.

To me you should include something like the Kaspersky Secure Network in CIS. Then when D+ will give an alert that a program wants unlimited access, Comodo can make use of the community to tell us when the file was first seen and how many users are using the product. This will help us to make decisions for unknown files.

IMO security with zero user intervention does not exist. Even Kaspersky prompts the user for action when a file not digitally signed is executed.

Of course you should keep improving the AV as well in order to maintain the reputation of Comodo. According to a recent survey conducted by AV-C, the majority of people still consider on demand tests as the most useful tests.lol

Regards

and that is the problem!

those tests cannot test how well a product like CIS protects against infection yet people rely on it…so go figure…

Defult deny work just fine, just make sure your settings are correct and disable the whitelist or else you will have a “default deny some of it” configuration.

Default deny should protect the system with default settings.

While I am glad that you got some replies from The Boss, I am still disappointed in the response, but it is good to receive a reply. :slight_smile:

I guess this is a Need-To-Know situation, but I do hope that CIS is tested one day and I do hope that it continues to improve, and that the Comodo Team improves the efficiency of their Lab responses; I imagine it is hard trying to deal with so many submissions, so finding more efficient ways would really help.

Need-To-Know defintion :smiley: for those that do not know :wink: :

I like how Avira allow you to Submit a file and it can show you a Result (slightly similar to CIMA but by just scanning it with the current database insteads of the CIMA approach) & E-mails it to you and/or lets you know it is Under Analysis & E-mails you when the Results are available; basically CIMA could be adjusted in this way and an automated e-mail system combined with Human Analysis when necessary would probably help.

Example of the Avira approach:

http://analysis.avira.com/samples/

I like how Microsoft also allows you to receive Automatic E-mail Results of Submissions, I think CIMA/The Web Submission Form/SiteInspector/DACS/Whatever combined/adjusted/modified/partially automated/improved/made more efficient/databased/etc would really help the Comodo Team and the Users if balanced properly.

Example of the Microsoft approach:

https://www.microsoft.com/security/portal/Submission/Submit.aspx

I think it would help with the response of False Positives and New Malware, and the time & resources needed to respond, and Users could at least get Automatic Responses (especially by e-mail) letting them know the clear/detailed Results of their submissions & they will at least know that someone or something looked at their submissions, and the Comodo Team could then spend more time dealing with the Unknowns instead of the Known knowns. :smiley:

All this data could be more efficiently stored/databased to help avoid Unnecessary Analysis of files that have already been checked, and can help put more focus on what is not known; and much of this data could be searchable by the Comodo Team & Users, without as much hassle & in a more unified way.

Then this data and partially automated system could help improve several Comodo Products and put less strain on the Team & Users (especially the automated response system, which could help the Moderators especially, and help the Users feel that their contributions actually matter/help).

Anyway, enough of my jibber jabber, but do have a look at the Avira & Microsoft approach, among others. :wink:

Thank you and keep up the good work Team Comodo. :slight_smile:

Maybe so, but would it hurt anyone to use Old & New testing methods, to help them feel better at least? :smiley:

I mean, it would not kill anyone would it? :smiley: :wink:

Also people could see a comparison of the Old & New testing methods, and maybe with time, they will see what you see. :wink:

But to deny them that, will make many Suspicious and their minds will probably stay closed, and they will probably fail to see what you see. :wink:

But what do I know? :smiley:

But is it right to mislead end users?

They think these old tests measure “protection”. Should we continue letting them believe that and fuel the misunderstanding?

The majority of end users directly associate these test results with the protection capability of the product being tested. Is it right to continue to fuel that misconception?

or is it time to stand up and educate?

I do not see anything wrong with trying to Help Educate people, but I think it is Wise to balance/adjust the Approach/Teaching Method to better Educate/Teach, depending on the Individual/Individuals. :wink:

It is not easy, but you have to find a Method or Methods that can better Reach a Variety of People or at least the Core/Main/Key/Mainstream Group. :wink:

So, show them a Better Alternative to The Old Testing Methods and give them an Alternative To AV-Comparatives/Etc., and the Mob May/Might follow/be educated/learn/etc. :wink:

Make a Detailed/Easy To Follow Template/Example of The New Testing Methods in Written/Picture/Audio, Video Form/Etc, maybe get some Volunteer Testers (Such as Languy) to make YouTube videos/etc, maybe make a simple testing program, etc.

Either way, once more people can clearly make sense of what you see and have clear examples to follow, then maybe more will join/learn/follow/be educated. :slight_smile:

But what do I know? :smiley:

Thank you for responding, show us the way, and maybe some of us can be Rehabilitated. :smiley:

You see there are few dynamic tests out there

http://www.melih.com/2010/10/12/i-hope-you-will-now-stop-spreading-misinformation-symantec/ (here is the one that was done by AV Test in Germany. )

and some more here Comodo Forum

and the biggest test of all is the end users. There are many many and MANY who come and tell us since using Comodo they didn’t get infection and so on… We are the MOST “Anti” Virus :slight_smile: . We stop them all, irrespective of whether we know them or not :slight_smile: Traditional AV can only stop the ones it recognises ( a BIIIIIGGGGG difference…)

And this is how we try to educate http://www.comodo.tv/home-computing/the-good-the-bad-the-unknown/ (261,000 views! and growing)…

and another one Virus Protection Vs Virus Cleaning - Comodo Blogs (245,000 views and growing)

And better alternative is www.AMTSO.org which is an organisation that is trying to standardise how one should test a security application and we are part of it.

thanks

Melih

Thank you for the URLs, I will try to check them out, it sounds like y’all may be on a correct/proper/better path. :wink: :slight_smile:

But, if the other AV has high detection rate compared to CAV, you will actually see less D+ alerts compared to using Full CIS suite, right?..

OK, I understand that you don’t want to get CAV tested by the AV-comparatives standard test because if it scores below the top AVs then uneducated users will assume CIS is worse than the other products.

Despite this, there are many people on and off this forum who would like to see how CAV compares with other AVs in AV-comparatives tests.

Any AV including CAV will miss some malware. Are you telling me that using CAV will cause fewer D+ alerts than using another AV with a better detection rate? If so, can you explain why this would occur.