What topic my friend, and it is true what I said and I do not want to be repetitive. Such a test is worthless if it’s not approved by the AMTSO review board. Actually it is the nitty gritty of that topic whether you like it or not.
Peace.
But Comodo as an entity only recognizes tests that are AMTSO approved. Is this one of them?
Peace.
Vendor approval is NOT a prerequisite of an independant test. If it were, the results would be automatically questionable as reviews would only be done if they met the criteria of the vendor.
I think it’s an error (Comodo missing from the image). Note NOD32’s score on the image, it’s Comodo’s 98.1% score rather than NOD32’s 98.3% on the table and it’s in the wrong position for NOD32. Also, NOD32 is No32 in table.
From their site about Comodo :
We used its Antivirus only with no proactive features turned on, in other words we performed a right-click scan on the sample package.We are finalizing the presentation, the test will be published today.
JAKI,
why your preoccupation with AMTSO Testing in your own words
I myself, as well as Melih and many others, want to see Comodo AMTSO tested, but you are using this as an opportunity to bash Comodo
X
The error in the image is corrected. 
http://malwareresearchgroup.com/wp-content/uploads/Sveta/2009/01/GenerateChart.jpg
How many Samples tested?
How many Samples the top one has missed?
How many Samples do you need for infection?
How many more new viruses/malware out there on top of the Sample set used?
Would be grateful if someone could answer this…thanks
Melih
We as users, most of them at least, are quick to take credit for a test result in order to show that CIS is making progress, that’s all fair and good. However, Comodo itself would not recognize such a test as valid. Comodo only recognizes tests that have been reviewed and approved by the AMTSO review board.
In that sense, I do not know if CIS is making progress at all. There is neither Westcoast labs, nor ICSA labs certifications, just plain nothing.
You said that I’m bashing Comodo. Please quote my posts where I made such a statement. I’m a man of my words and there is no doubt that I will be able to own my own statements.
Peace.
Hi Melih
Would it matter? What difference would it make when the test is not AMTSO approved? Or is it? ;D
Peace.
How is it that I’m bashing when I’m pointing out something factual? Do facts hurt you this much?
Peace.
554 891
How many Samples the top one has missed?≈ 1 110
How many Samples do you need for infection?1 :P
How many more new viruses/malware out there on top of the Sample set used?Unknown… many.
Would be grateful if someone could answer this..thanksMelih
Tried.
Comodo may not recognize these tests…but that doesn’t mean we shouldn’t. Although not totally accurate, they can still be an insight. I certainly wouldn’t choose Norton over McAfee just if Norton got 98.99% on some test and McAfee 98.98%.
But if Norton got 98.99% and BesterestAntijunk got 12% … I would have a higher opinion of Norton than BesterestAntijunk.
I do not feel like Jaki is “bashing”.
I’m not sure how this was meant…but AMTSO is not the one who decides if a test is absolutely wonderful or worthless.
Hey…languy’s reviews are not “approved” by AMTSO, but I enjoy them, and feel like I gain from them.
Worthless, in terms of its importance. The AMTSO review board must review the test in order for the testing organization to get the AMTSO nod or seal of approval, sort of. Please do not believe me, believe Melih. He is the one who brought that fact to my attention. What can I say, he was right.
Peace.
Insight to where. Only an AMTSO test that is approved by its review board would divulge the efficacy of a security product like CIS for example.
Peace.
Bingo…
so even missing 1 is not acceptable if you want protection!
Melih
G’day,
IMHO, this has highlighted a serious flaw in the AMTSO process.
If AMTSO was set up to produce guidelines on how to dynamically test antimalware products in a real-world environment, they should have included the condition that the testing organisation should have to submit their testing process to AMTSO for review and certification PRIOR to running the test and announcing results.
Membership in an organisation does not guarantee conformity to the published principles and intent of said organisation. Review and certification are essential in safeguarding the integrity of any results.
Ewen
I could not have agreed more. So has MRG test been reviewed and certified?
Peace.
from http://malwareresearchgroup.com/?page_id=2 Amount of samples used in this test: 554.891Malware categories used in this test and the amount of samples in each category:
Trojans/Backdoors- 398.951
Windows Viruses- 8.864
Worms- 61.928
Adware/Spyware- 48.552
Rootkits/Exploits- 10.736
Other Malware- 25.860
How many “false positives” ?
http://malwareresearchgroup.com/?cat=3 Making this review we used the latest version of COMODO Internet Security (3.12.111745.560), Database Version: 2470.Reviewing process had three stages:
On Demand scan on 50.000 samples of malware (June,July,August and September)
Self Protection test where we used various tools with which we tried to disable COMODO Internet Security and its services.
System Protection test – COMODO Internet Security was tested in Real Time against various most dangerous malware samples (better known as System Killers)
Result of our reviewing process:
On Demand scan test – COMODO Internet Security failed to detect 153 samples of malware out of 50.000, scoring a detection rate of 99.69%.
Self Protection Test – COMODO Internet Security successfully blocked all 10 attempts to disable it and its services.
System Protection Test – COMODO Internet Security successfully detected and blocked all 15 System Killers leaving the system unharmed and fully operational.
Conclusion:
COMODO Internet Security offers outstanding level of protection…
No. But then again, they’re not claiming to be AMTSO compliant. This is a detection test with the added ding-■■■■ of termination protection testing (which all tests should do BTW, IMHO, OK ;)).
Current AV testing methods and AMTSO testing methods are apples and oranges - it’s hard to draw a comparison between them. One starts from a position of “assumed dirty” and the other start from a position of “assumed clean”.
Cheers,
Ewen
P.S. Nice result, though.