Detection rate comparatives about CIS and 3rd party final AV products by darcjrt

darcjrt · January 25, 2009, 3:30pm

Thanks Be creative and forget about my last post. I might add more apps to compare!

Thanks!!

Melih · January 25, 2009, 3:32pm

in order to calculate the detection rate, we would need the exact no of malware samples you have and how many malware samples are left undetected by the relevant AV engine. We should not rely on what AV reports as it found as it could be counting things in a different manner…

So it would be great to have “exact no of samples” and how many “samples AV doesn’t detect” (rather than relying on AVs to tell us how many they detected).

thanks for this great work!

Melih

darcjrt · January 25, 2009, 3:40pm

I will run a counter on the infected folder and try to in some way detect and count with as much as tools as possible malware around the HDD. I have some ways!

I will stop adding malware at this point to get an exact number.

Please if you have any other suggestion please let me know. I want this to be as close to reality as possible!

Thanks Melih!

Melih · January 25, 2009, 3:51pm

here is a new excel with the relevant fields added and detection ratio calculation formula done…

all you need is to inject the numbers for the fields “undetected” and “sample size” the detection ratio will be calculated automatically.

thank you

Melih

[attachment deleted by admin]

darcjrt · January 25, 2009, 4:14pm

(:CLP)

THX

alaertsxan · January 25, 2009, 4:22pm

Topic name changed … Like it ?

Xan

darcjrt · January 25, 2009, 4:37pm

Love it!

THX!

alaertsxan · January 25, 2009, 4:42pm

There is one thing I would like to ask you, could you edit your first post and add a warning that you’re not an official tester, but an independant home user ?

Thanks,

Xan

Melih · January 25, 2009, 4:52pm

hehe… what is an official tester? There are no official testers…

Melih

alaertsxan · January 25, 2009, 5:09pm

non affiliated tester then ?

Xan

darcjrt · January 25, 2009, 5:16pm

haha

Its Ok. I do this for 1. Fun 2. track CIS detection rate.

Will edit right now

Melih · January 25, 2009, 5:26pm

hehe… sure…

Melih

alaertsxan · January 25, 2009, 5:48pm

What ? AFAIK most, if not all, are connected to companies, how do they get the samples do you think 88)

Xan

Melih · January 25, 2009, 5:59pm

thats so very true!!!

Citizen_K · January 25, 2009, 6:02pm

I see different numbers for each scanner under Scanned even though they all scan the same amount of files. What is the definition of Scanned? Is it something you define and if so how is it defined? Or is it what the scanners report they have scanned?

CIS doesn’t scan files bigger than 20 MB by default is that of influence on the results?

Kudos for all the hard work. (:CLP):■■■■

darcjrt · January 25, 2009, 11:13pm

CIS is set to scan files up to 999MB.
I removed the option to scan memory on every scanner. So I have to exact answer on why a scanner reports more files than others.

Attached the official scan results AM. Posts before this one are BM.

AM = After Melih’s Spreadsheet
BM = Before Melih’s Spreadsheet

Hope you enjoy the results.

[attachment deleted by admin]

Kyle142 · January 25, 2009, 11:34pm

Hey darcjrt, Why is it that you disable things such as heuristics and memory scanners on other products? That isn’t fair

triple · January 25, 2009, 11:39pm

30% difference is a lot!.. though it is less than what I said before… (~ 40%)

jeremysbost · January 26, 2009, 1:28am

I think he disabled it for all programs, including CIS. That seems sorta fair… :THNK

darcjrt · January 26, 2009, 2:01am

Indeed…all of them.

I’m testing detection rate not removal capacities. I don’t care about memory right now and I also counted every single piece of malware on the HDD to get an exact number for the spreadsheet.