PC Security Labs July Test

[b]Here is the link to the final results on the official website of PC Security Labs.

http://www.pcsecuritylabs.net/

-Lan[/b]

Hello All, I have finished the new report July 2010, now it is time to make preparation of September one and Cleaning test

Chinese report: http://www.pcsecuritylabs.net/document/report/PCSL_Total_Protection_Test_July_2010_CN.pdf

English report: http://www.pcsecuritylabs.net/document/report/PCSL_Total_Protection_Test_July_2010_EN.pdf

PCHome report: PCSL全方位安全软件测试2010.7月报告-PChome

Regards
Jeffrey

I did a couple of questions:
Could you explain the methodology?
Was the malware not detected in the dynamic test able to infect the computer?

For how to deal with sandbox in dyanmic testing, I have explained clearly to umesh[at]comodo who is in charge of comodo security part.

The malware can bypass sandbox even if the client says the malware is quarantined or tells you needn’t worry about it.

None of the intelligent solution can block 100% of the malware.

Nice Question, thank you for your consideration

In the future please don’t link to a wilders page. When the test is officially available we mods will link to the official testing groups page and make a sticky of it. Thanks.

Ok sorry, I didnt know that, I wanted to post relevant information being discussed in wilders like there is malware able to bypass the sandbox.


This is what they said

Here is ok. Some of the malware drops PE out of sandbox onto real machine. I have given this feedback to comodo and I think they will improve the sandbox function.

I have never seen malware drop out of the sandbox. Also it is a little fishy that Jiangmin did so good. The reason I say fishy is that I believe most of their malware samples are ones that are prevalent in the asian market, so that means AV vendors that are based in the asian market will see those samples first compared to companies located in other parts of the world. I would hope that they use malware that is a good representation of the whole world not just one part of it. But this is just my opinion of it based on personal experience.

I havent seen a malware bypass the sandbox lately but seems that PCSL found some and they have reported the problem to comodo so comodo can fix it, they are not hiding nothing.

also I think they are mistaking dropped file for a real system infection. Remember v4 had many more dropped files onto the hard drive compared to V5 and if you didn’t know how the comodo sandbox specifically works you would assume that the system was infected even though it is not.

I asked about this in wilders forums, if the PE files remain active or not, lets see what they say.
Probably the files are not active = the computer is not infected…

Anyway with this new feedback Comodo should improve the sandbox.

Anyway even if Comodo is able to monitor every single file and registry change created by the application sandboxed, should be everything be deleted? what if is not a malware…

more importantly look at the dynamic detection detection, check out how many comodo catches dynamically and also look at the FP’s not that high especially compared to others.

But if the PE files are not active in the computer ergo the computer is not infected, the dynamic detection should be 100% .

well I don’t know how they counted the dynamic detection, did they just count the AV pop ups, the HIPS pop ups, sandbox? Their test does not say how they counted dynamic detection, we need more information on how they came up with their numbers.

Maybe Umesh can tell us something else… lets wait
Or send to him a PM about this thread

Although I said Anti Virus product on its own can’t protect you, its good to see the 3rd party confirmations of the good work our AV Labs have done in getting us a top notch detection ratio.

Melih

Yep, also would be nice to see the cloud behabiour blocker and the cloud AV in this test (CIS 5) the % would have been much higher

indeed! all in good time ;)…there is more than just the Cloud based AV, Cloud based Behaviour Blocker…in subsequent releases to v5…:slight_smile:

Melih

For instance? …

Super CLoud mega scanner? ;D what’s new in CIS after v5? please tell us something jejeje

Congratulations to a 98.46 detection score! Not the best score of those tested - but CIS is improving!!!

Version 5 with the cloud thing will hopefully take CIS even further in upcoming AV-tests.

yeah 98.5% is amazing considering this AV engine is only about 2 years old. The rest of the others are much older.

Hi lordraiden,

As of now i have yet not discussed Dynamic Detection part with Jeffrey wrt to results, but in general he stated following about methodology:

For HIPS test, mostly we make the detection based on the notification of the HIPS module, e.g. if the HIPS notify the users there is a malicious behavior and users should make a deny decision, we then deny the execution. When there is no clear notification that show the behavior is malicious or dangerous, we make the allow decision. And the final decision is whether there is an infection to the PC(cause data loss, infection of system files, etc. ) We make this also in our Total protection test.
For sandbox, in dynamic block test, if it automatically executes in sandbox and no files leak out of the sandbox, then it is a successful block. For dynamic test, if it is automatically sandboxed, then it is regarded as a fp and if the window with three options: sandbox, allow, deny. Then we choose allow and then see if the client block the installation of normal software.

So far my discussion with Jeffrey has been limited to his definition of false-positive wrt Sandbox. So i have yet to gather information from Jeffrey if he has encountered any malware which by passed Sandbox and was able to infect system in the sense that it was able to persist.

In general as you know Sandbox doesn’t allow to drop files in privileged folders, like Windows or start up folders.

Thanks
-umesh