CAV3 detection rate test [2008.10.05] - 10.26% [CLOSED]

A short test conducted over a period of nine days. Samples collected from various sources, and all hand-verified to be malicious using behavioral analysis and (some) disassembly. All undetected samples have been forwarded to Comodo using the inbuilt submission system.

DATE SAMPLE COUNT DETECTION COUNT DETECTION RATE (%)
2008.09.01 80 2 2.50
2008.09.02 268 35 13.06
2008.09.03 118 13 11.02
2008.09.04 190 12 6.32
2008.09.05 138 34 24.64
2008.09.07 167 14 8.38
2008.09.08 66 6 9.10
2008.09.09 109 14 12.84
2008.09.10 136 4 2.94

2008.10.02

  • total samples: 67
  • detected: 4
  • detection rate: 5.97%

2008.10.03

  • total samples: 183
  • detected: 8
  • detection rate: 4.37%

2008.10.04

  • total samples: 61
  • detected: 0
  • detection rate: 0.00%

2008.10.05

  • total samples: 39
  • detected: 4
  • detection rate: 10.26%

Thanks Solcroft (:HUG) You deserved a cookie (:TNG)

COMODO will need a lot of time to build a respectful malware database.

Hi scolcroft.

Can you please send the malware samples using these instructions.

Thanks!
Josh

12 months according to Melih, then they will be on the top :slight_smile:

But he can’t do it by himself. we all have to help out and send in all the samples we can find

Yes. And off course, Backed up be a dedicated team. :slight_smile: (As you can see many Staff of Comodo are around!) :slight_smile: You all have a major role too! :slight_smile:

Josh

Yep
This is the best way to enlarge the database

If you consider that a file is suspicious, and CAV 3 is not currently detecting it,

  1. Please zip up (using a archive tool like winzip, winrar etc) the file that you believe is wrongly detected and password it with password ‘infected’ without the quotes and email it to malwaresubmit at avlab.comodo.com

  2. Please make sure to mention “SUSPICIOUS FILE SUBMISSION” on the subject line of the mail.

By the way how many people are there over at Comodo to do the malware research?

I don’t want to be a party pooper, but thats really just a wishful thinking.

No need to ruin the mood RejZor :slight_smile: Will be, won’t be - time will tell. There haven’t been any official testing yet, hence CAVS 3 is in beta stage. I hope it won’t be stuck there as CAVS 2 and we’ll see how COMODO performs on test labs.

Though as Melih said - no AV test can be considered accurate :slight_smile:

But you can clearly see the difference when antivirus A, B, C and D are detecting >90% of malware and antivirus E is detecting just 40%… (just example). Latest AV-Comparatives was using over 2.000.000 samples. If you fail at that there is clearly something terribly wrong, accurate or not…

Agreed. :slight_smile: Lets wait till CAVS 3 comes out of beta and gets tested. After a few months we’ll see if it improves or not :slight_smile:

All that means is the malware being tested is caught by that specific malware. How about if I can show you hundreds of malware that only we catch at a certain point that noone else catch?
Will you then stop believing that they are not good?

Of course not!

Just because some malware you have is detected by a specific AV does not mean that they are able to detect some other malware that will infect someone else around the world!

Anyway, our job is to protect our users! We do a good job on that. AV is a secondary defense layer and within 12 months we will be one of the best. That is what we will spend our money in. To protect our users! I hope everyone can join in our efforts to help us achieve that!

RejZoR here is a question for you: If one of the AV testing guys gave us all his malware list, then the world would have an amazingly fast AV that, according to tests :), would detect 100%. So why don’t you and everyone concentrate in emailing these guys who keep the malware library to themselves and get them to share their malware library with us so that we can protect YOU and everyone?

Don’t you think if you did that, it would be a great service to human race in general? Why don’t you do that? Why doesn’t everyone do that? Its a simple solution!

Melih

Here:

https://forums.comodo.com/general_discussion_off_topic_anything_and_everything/open_letter_to_andreas_clementi_httpwwwavcomparativesorg-t27537.0.html

Its a very simple solution: Ask them to give us their malware library.

  1. The malware library is not copyrighted as such (eg: not their code)
  2. The direct beneficiary will be millions of end users because this is a free product
  3. Why hide a nasty code that could cause a damage to end users from very people who want to protect them?

So why would’t they give us the malware library? Who would benefit from not allowing us to protect millions of users?

Melih

Don’t let them wind you up Melih.

Combination of the whole CIS package and for me the inclusion of CBoClean running by it’s side is enough to protect my Production Machine.

P.S. I think our IT bloke is getting fed up with me talking about Comodo products, hell, I think everyone in the office is but that’s just me. I’ve got trust in the protection Comodo Products provide.

E

Sorry, i don’t have anything against Comodo (in fact you do have some really great programs like CRC and CPF), but this is just a very bad excuse for CAVS/CIS. You could really hardly target specific antiviruses with database consisted out of over 2,3 milion samples. Besides, i have no reason not to believe that Andreas that is completelly independent.
I see no reason why would he team up with any specific antivirus. In fact those who did not agree with him simply requested that their AV is not included in that test. Meaning they either really target just gateways/corporate segment (ex Fortinet, SOPHOS) or they just plain suck. CIS could work 8 years ago where malware was not this tricky. But today it just doesn’t seem to do much. Saying it’ll be better next yer is nothing. You an improve it against today’s standards but next year the situation may be completelly different and you’ll need next 12 months again.
Lagging behind this much all the time is just not good. And we all know antiviruses lag behind by default.
Some more, some less.

Defense+ is a also good idea with bad implimentation. If i were you, i’d head towards ThreatFire way, not classic/enhanced HIPS. They just don’t work and are aimed at strict geeks that want to keep every file under microscope like total paranoid freaks. I’m and enthusiast user and i prefered ThreatFire over anything else anytime you want.

Ask yourself the question, would it not be weird for a very large anti-malware vendor to share his massive database of nasty code with a competitive party that offers product for free, when the commercial has loads of paying customers who are paying for the exact same code ? ??? huh? ???

$$ is at the heart of it and it directly contradicts with the security of end users!

If testing guys were to give out their malware… then their testing the way its done today would no longer exist as everyone would get 100%. So its a self-preservation which is again goes totally against security of end users!

Melih

Well with the heuristics, people sending in suspicious files found and the malware lab people at work Comodo’s definitions should be up to par soon. As it is now its fine for a free program anyway its not like we are paying for it.

I am confident about catching the latest malware within a reasonable time.

However, testing organisations test using malware that is no longer around. So that forces AV vendors to go underground and try to find people who has collected libraries and sometimes people get paid for providing malware libraries! Which in turn encourages new malware because now you can make money by creating new malware and sell it to AV vendors who want to increase their detection!

You see, its a “broken system”! And we have people who feed that animal further by so called “testing”…

In my opinion the industry must get together and work in collaboration against malware! At the moment its too disjointed and hence malware is winning the war!

Melih