CIS RC2 passes all leak tests on Windows 7 RC

In my own testing CIS RC2 passed all leak tests, termination tests and other tests in the Security Software Testing Suite on Windows 7. ;D

All threats were warned against in clearly worded red or orange Alerts by CIS.

The only weakness was in Level 3: Using kill3f.exe CIS would pass, if you ‘Block’ the very clear alert regarding modifying the user interface of cfp.exe. But it would fail, if kill3f is allowed: cfp.exe would crash and then be vulnerable to wallbreaker and others. see also http://www.matousec.com/projects/proactive-security-challenge/level.php?num=3#result-table

Here my notes on the results:

Comodo Firewall RC 2 (3.9.76291.502) on Windows 7 RC with UAC

all tests were run from an elevated command prompt (as administrator)

SETTINGS:
Firewall: custom policy mode
Alert Settings: very high
Defense+: safe mode
Configuration: proactive security
Avira AntiVir: is set to ignore the ‘Security Software Testing Suite’ test folder (it would otherwise detect and block a lot of the tests)

LEGEND:
pass = passed the test when selecting Block in pop-up
crashed - the test app crashed (not CIS), no leak was detected
error - test app terminated with an error, no leak was detected

(P) = passed the test even after choosing ‘Allow’ in the CIS Alert
(F) = failed the test only after choosing ‘Allow’ in the CIS Alert
(?) = not tested by choosing ‘Allow’ in the CIS Alert

— RESULTS —

LEVEL 1
breakout2.exe - crashed
coat.exe - crashed
echotest.exe - pass (P)
kill1.exe - pass (P)
kill2.exe - pass (P)
leaktest.exe - pass (F)
tooleaky.exe - pass (F)
wallbreaker1.exe - pass (F)
yalta.exe - pass (F)

LEVEL 2
awft1.exe - pass (P)
dnstest.exe - pass (P)
ghost.exe - pass (F)
jumper.exe - pass (?)
kill3.exe - pass (P)
kill3b.exe - pass (P)
kill6.exe - pass (P)
wallbreaker3.exe - pass (F)
wallbreaker4.exe - pass (P)

LEVEL 3
awft3.exe - pass (F)
awft4.exe - pass (P)
dnstester.exe - pass (P)
kernel1.exe - pass (F)
kill3f.exe - pass (FAIL, if allowed - cfp.exe would crash and then be vulnerable to wallbreaker and others)
kill4.exe - pass (P)
kill7.exe - pass (P)
sss2.exe - pass (P)
suspend1.exe - pass (P)
thermite.exe - pass (F)

LEVEL 4
copycat.exe - crashed
cpil.exe - pass (F)
cpilsuite1.exe - pass (P)
kernel1b.exe - pass (?)
keylog1.exe - pass (F)
kill3e.exe - pass (P)
kill8.exe - pass (P)
kill9.exe - pass (P)
sss.exe - pass (P)
suspend2.exe - pass (P)

LEVEL 5
breakout1.exe - pass (P)
cpilsuite2.exe - pass (F)
crash1.exe - pass (P)
crash2.exe - pass (P)
crash3.exe - pass (P)
crash4.exe - pass (P)
kernel2.exe - pass (F) - the CIS-Alert could be misleading, as ThreatGuard gave a 99% approval rating, making it likely for a user to allow the threat
kernel3.exe - pass (P)
keylog2.exe - pass (F)
kill3c.exe - pass (P)
kill3d.exe - pass (P)
vbstest.exe - pass (F)

LEVEL 6
cpilsuite3.exe - pass (P)
crash5.exe - pass (P)
crash6.exe - pass (P)
ddetest.exe - pass (F)
echotest2.exe - pass (F)
firehole.exe - pass (P)
flank.exe - crashed
kernel4.exe - pass (P)
keylog3.exe - crashed
keylog4.exe - crashed
kill10.exe - pass (P)
kill11.exe - pass (P)
runner.exe - crashed

LEVEL 7
bitstest.exe - crashed
firehole2.exe - pass (P)
keylog5.exe - pass (F)
keylog6.exe - crashed
kill12.exe - pass (P)
osfwbypass.exe - crashed
runner2.exe - crashed
schedtest.exe - crashed
sss3.exe - pass (P)

LEVEL 8
kernel4b.exe - pass (P)
kernel5.exe - error
keylog7.exe - crashed
kill5.exe - pass (P)
newclass.exe - crashed
schedtest2.exe - crashed
socksnif.exe - error
sss4.exe - pass (P)

LEVEL 9
crash7.exe - PASS (P)

LEVEL 10
not tested

Me too.

I am so happy.

CIS failed only 2 and passed all other leaktests.

Congratulations!

Andreas

[attachment deleted by admin]

just thought i should say that cfp.exe is the GUI for CIS, cmdagent.exe is what protects your PC and even if cmdagent.exe is down you still have the CIS drivers loaded (i think they protect your PC also if cmdagent.exe is down (not sure about this))

Also try proactive security mode, from all the tests i have done nothing can get passed CIS 3.9 in Proactive Security

Lots of bugfixes were added in this version. Would it be safe to say that whatever caused Online Armor to top in Matousec tests is now properly handled?

the tested products are installed on Windows XP Service Pack 3 with Internet Explorer 6.0 set as the default browser. IE6 browser with faults ;D

OmeletGuy, yes I used Proactive Security, as stated in ‘settings’. kill3f was detected by CIS, so it passed the test in regards to cfp.exe. I just wanted to see, if it could crash cfp.exe, if I allow it. It crashed, after hammering it for a number of seconds. If I had checked “Block all unknown requests if app is closed”, all other leaktests would have been blocked anyways.

Searinox, my setup was different from Matousec’s. He used XP SP3, whereas I wanted to test specifically Windows 7RC with the latest CIS RC2 (.502), and had the exact same results in Windows 7 as in Vista. Some of the tests did not execute properly in Windows 7 (and in Vista), making these OS more secure, until those tests or similar exploits are updated. Thus I could not reproduce any of the failures in Matousec’s report. Anyways, if you look at his detailed report, you will find that none of the tests, which CIS failed are particularly relevant for user security.

My installation only included Firewall and Defense+, without the CIS-AV. Therefore I could not find any process called “cssurf.exe” (which Matousec terminated with Kill3f.exe). This process seems to be irrelevant for the tested HIPS security anyways.

CIS did not fail the SSS test and unwanted user logout was prevented, probably due to improved OS security. CIS also passed the SockSnif test under the setup, but socksniff failed with an error message which I suspect is related to a Vista/Win7 incompatibility of the test.

Andreas, actually, it did not fail any. It passed 100% on Windows 7RC.

XP SP3? Oy. People still use that antiquity to determine the standards of security?