Welcome, Guest. Please login or register.
November 22, 2009, 11:18:23 PM

Login with username, password and session length

336961 Posts
37290 Topics
84542 Members

Latest Member: papagino

Search:     Advanced search | Tag Cloud
+  Welcome to the Comodo Forum
|-+  Desktop Security Products
| |-+  Comodo Internet Security - CIS
| | |-+  Leak Testing/Attacks/Vulnerability Research
| | | |-+  New Matousec Firewall Challenge
« previous next »
Pages: 1 [2] 3 4 5 Go Down Print
Author Topic: New Matousec Firewall Challenge  (Read 16803 times)
Kyle
Computer Security Testing Group
Comodo's Hero
*****
Offline Offline

Posts: 3126



WWW
« Reply #15 on: November 29, 2008, 12:47:08 AM »

Hey, I thought that Comodo weren't going to participate in those tests because some of the tests were irelevant in real world situations, Has Melih changed his mind or as Matousec changed his methods?
Logged

E5200 2.5ghz [at] 3.2ghz, POV 9800gt 512mb, 2gb DDR2 RAM.  500gb. HDD

DUAL BOOT: 
Linux Mint - Everything.
Win XP - Only when needed.
Goose19
Comodo's Hero
*****
Offline Offline

Posts: 1218



« Reply #16 on: November 29, 2008, 12:50:28 AM »

Well seeing how alot of people read these "tests" and decide what they will use by these tests only I could see why Comodo would like to be #1 or the top 3 like they have been for so long.  Grin
Logged

System Specs:  Pentium 4 with HT 3.06 Ghz,  1.5GB RAM, 160 GB WDC HD, Nvidia Geforce 7600GT 256MB DDR3



New Build: AMD Athlon 64 x2 6000 3.1 Ghz  4 Gb RAM 320GB WDC Hard Drive 650 watt quad rail Power supply(overkill Cheesy) 9500GT Hybrid SLi with 8200 (onboard video) Decent Gaming rig Smiley
3xist
Guest
« Reply #17 on: November 29, 2008, 12:55:20 AM »

There is a reason... But  Lips Sealed

Josh
Logged
MagisDing
Comodo Family Member
***
Offline Offline

Posts: 54


« Reply #18 on: November 29, 2008, 03:00:14 AM »

There is a reason... But  Lips Sealed

Josh

what reason? Thinking
Logged
hullboy
Comodo Loves me
****
Offline Offline

Posts: 172



« Reply #19 on: November 29, 2008, 03:48:18 AM »

Hey, I thought that Comodo weren't going to participate in those tests because some of the tests were irelevant in real world situations, Has Melih changed his mind or as Matousec changed his methods?

As we have seen, participating in this kind of tests is very useful otherwise the bug wouldn't have come out (B)
Logged

Windows XP Home SP3 32bit
NOD32 2.70.39
COMODO CIS 3.13.120417.573
Configuration: Proactive Security
Firewall Security Level: Custom Policy Mode
Defense+ Security Level: Clean PC Mode
TerryWood
Comodo Loves me
****
Offline Offline

Posts: 100


« Reply #20 on: November 29, 2008, 06:34:21 AM »

Hi All

Being a long time user of Comodo Firewall, and supporter of Comodo, it is both disappointing and disturbing that Comodo is well down the lists in the latest Matousec challenge.

Some of the small organisations appear to be running rings round Comodo, in that their submissions appear to be fully checked for workability and compliance.

We have the strange situation where Comodo is developing a suite of leak tests that are supposedly onerous and yet Comodo cannot get near pole position in the Matousec tests? A bug we are told is the problem (pretty quickly after Matousecs tests were released). Its a pity that this was not determined before the test.

If the Comodo brand (Yes it is a brand, and potentially a strong one at that) is to mean anything to the loyal users who use it and create the brand strength, then the brand has to have credibility. I am surprised that Melih is not spitting blood just as Comodo's competitors will be laughing up their sleeves.

It really is not good enough, it is sloppiness of a high order, and does not bode well for Comodos commercial aspirations.

Yes I am sure there will be plenty of forgiveness about in this forum for those closer to the Comodo organisation. I am not sure that is what is needed though. 

Terry
Logged
nick55555
Comodo Family Member
***
Offline Offline

Posts: 83


« Reply #21 on: November 29, 2008, 09:16:22 AM »

The latest Firewall Challenge tests on matousec.com are a bit disappointing. CIS fails on many crash and termination tests...   Sad
Logged
Kyle
Computer Security Testing Group
Comodo's Hero
*****
Offline Offline

Posts: 3126



WWW
« Reply #22 on: November 29, 2008, 09:25:57 AM »

The latest Firewall Challenge tests on matousec.com are a bit disappointing. CIS fails on many crash and termination tests...   Sad

Hey Nick, Please do a little bit of reading on this thread - it will explain that there was a bug.
Logged

E5200 2.5ghz [at] 3.2ghz, POV 9800gt 512mb, 2gb DDR2 RAM.  500gb. HDD

DUAL BOOT: 
Linux Mint - Everything.
Win XP - Only when needed.
Star Shadow
Computer Security Testing Group
Comodo's Hero
*****
Offline Offline

Posts: 286


« Reply #23 on: November 29, 2008, 10:13:50 AM »

Hi All

Being a long time user of Comodo Firewall, and supporter of Comodo, it is both disappointing and disturbing that Comodo is well down the lists in the latest Matousec challenge.

Some of the small organisations appear to be running rings round Comodo, in that their submissions appear to be fully checked for workability and compliance.

We have the strange situation where Comodo is developing a suite of leak tests that are supposedly onerous and yet Comodo cannot get near pole position in the Matousec tests? A bug we are told is the problem (pretty quickly after Matousecs tests were released). Its a pity that this was not determined before the test.

If the Comodo brand (Yes it is a brand, and potentially a strong one at that) is to mean anything to the loyal users who use it and create the brand strength, then the brand has to have credibility. I am surprised that Melih is not spitting blood just as Comodo's competitors will be laughing up their sleeves.

It really is not good enough, it is sloppiness of a high order, and does not bode well for Comodos commercial aspirations.

Yes I am sure there will be plenty of forgiveness about in this forum for those closer to the Comodo organisation. I am not sure that is what is needed though. 

Terry
There are more tests in the Matousec tests. Some of them actually can damage the system if I read it correctly. So, I don't think people want to use those tests. Tongue
Logged

I'm getting Married!!!
larsson
Comodo Member
**
Offline Offline

Posts: 27



« Reply #24 on: November 29, 2008, 10:42:26 AM »

I notice that this test is not a complete new test. Online Armor´s free FW result is dated 15 of May and thus made with previous test setup...

I think this test should be done with the same test setup as ranking otherwise might not be correct
Logged

XP 32bit SP3, CIS 3.13, SAS pro and Secunia PSI
AMD 64x2 6000+
egemen
Administrator
Comodo's Hero
*****
Offline Offline

Posts: 2134



« Reply #25 on: November 29, 2008, 11:22:29 AM »

Hi Guys,

We have reviewed the results. Dont worry. There are 3 issues to be highlighted here.

1 - There is a single bug in CIS/CFP which affects the previsous versions too.
This bug allows CFP/CIS to fail to detect a special type of handle duplication operation. And thats why it cause CFP/CIS to fails the following tests:

Kill1.exe kill2.exe:  kill9.exe kill12.exe crash1.exe crash2.exe crash3.exe crash4.exe crash5.exe crash6.exe

Do not worry. We have fixed the bug and you will be receiving the update on Tuesday(2/12/2008).

2 - There are errors in the test report:

CFP/CIS does not fail the following tests: kill3b.exe kill3f.exe kill3e.exe kill5.exe SSS2.exe SSS3.exe

It is unclear why CFP is reported as failed in these tests. You might try yourself and see. Attachment contains ssts.conf file with which you can test CFP/CIS.

CFP/CIS intercepts system shutdown privilege elevation requests and hence effective blocks sss2.exe and sss3.exe tests.

CFP/CIS can not be terminated by any of those kill tests.
3 - There are some insignificant tests that do not pose any real threat and hence we will not do anything about them.

--------------
CFP/CIS is marked as failed in the following tests:   SSS.exe i.e. System Shutdown Simulation tests.
--------------
It has been scored 50% because CFP/CIS does not intercept system shutdown requests. This is the testing methodology of the tester.

System shutdown poses no real threat. The malware waits for system shutdown to perform its harmful actions. So whether you intercept or not, it can attack the user when the user manually logs out. Original System Shutdown Simulation tests do care about this fact.
So we do not plan to add this redundant protection to pass any tests.

--------------
socksnif.exe: This test is designed to test if a malware can snif your network connection or not.
--------------

If "\Device\Afd\EndPoint" is added to the Defense+ My Protected Files list, this can be easily intercepted. Adding this entry to the protected files means, making Defense+ to alert you for each and every application which tries to access Windows Sockets.

However, we do not plan to add this to our default protected files. Because

* It poses no real threat. Malware can not sniff your everyday bank transactions because everything is already SSL encrypted,
* This is basically no different than sniffing your network traffic from another computer,
* It will increase the number of popups unnecessarily,

--------------
crash7.exe: This test tries to allocate all the memory of the computer to crash applications including the security software
--------------

It might be possible for an application to crash if there is no more computer memory available. This is usually a random case. We do not plan to make any changes to pass this test because

* The crash can be random, intermittent and ubiquitous
* Assuming CFP/CIS processes also crashed, there is no real threat to the system because by terminating CFP/CIS, malware will not gain any advantage for byapssing Defense+.

So in summary: by terminating CFP/CIS, Defense+ will not be able to be bypassed.
« Last Edit: November 29, 2008, 08:27:01 PM by egemen » Logged
lordraiden
Computer Security Testing Group
Comodo Member
*****
Offline Offline

Posts: 30


« Reply #26 on: November 29, 2008, 11:32:26 AM »

You can reply exactly the text of your post to Matousec for improve the Suite Test and the methology
Logged
Melih
Comodo's Hero
Administrator
Comodo's Hero
*****
Offline Offline

Posts: 8246



WWW
« Reply #27 on: November 29, 2008, 12:18:16 PM »

We hope David/Matousec will retest and update his results with our new version with the bugs fixed.

It would be very suspicious if he chose not to retest the bug fixed version, knowing that he made an error on his tests. It would show a goodwill from his side, we'll see if he does that or not...

lets watch out..


Melih
Logged

Melih
Comodo's Hero
Administrator
Comodo's Hero
*****
Offline Offline

Posts: 8246



WWW
« Reply #28 on: November 29, 2008, 12:27:56 PM »

Just to re-iterate the point:

We always had the security of the user in mind and CIS does have all the protection code built in to protect the user! That is our no 1 priority, always has been, always will be! We invented User Security in the world of false sense of security with AV companies giving products that does not protect end users from malware they can't detect, and we are the guys who put prevention as first line of defense into a mass product!

We will continue to protect the users and give them the best possible security solution, period!

We will NOT be guided by some leak test website for our development nor will be pushed into developing code just to pass these leak tests if we don't feel it will offer any benefit to our users!

To summarise: You get the best possible protection with CIS! You always had, you always will!

Melih
Logged

Star Shadow
Computer Security Testing Group
Comodo's Hero
*****
Offline Offline

Posts: 286


« Reply #29 on: November 29, 2008, 01:37:51 PM »

Melih, thank you for keeping us informed as to what is going on. I know you don't let testing sites guide your development, but I was wondering if you have been in contact with David/Matousec and team to see why they labeled failures when you said that CFP does not fail those tests. I think it is important for testing sites to at least have accurate results posted, so it shows that they are not unbiased. Even if they do not retest the bug fix version and amend the results for the version they did test, it would at least put Comodo higher in the list, and thus users that do not understand anything about testing sites will see Comodo as a better product, and for the vendor comments section, you can post that there is a big fixed version and also do your own testing with the Matousec test suite and state what the score is with the bug fixed version for the comment section as well. Smiley That would show other people how serious you are about security. We all know as forum members, but other people that do not visit these forums do not know, so that comment section is the place where you really need to show it. I know a lot of people that base everything on testing sites, so the comments are very important. Smiley I think egemen's post is really good and would benefit a lot of people being posted on the Matousec site since it shows how fast bugs are taken care of and how committed everyone here is.

Keep up the great work all. Smiley
Logged

I'm getting Married!!!
Tags:
Pages: 1 [2] 3 4 5 Go Up Print 
« previous next »
Jump to:  

SSL Certificate Free Virus Removal Firewall
Page created in 0.074 seconds with 19 queries.
Powered by SMF 1.1.10 | SMF © 2006, Simple Machines LLC
Seo4Smf v0.2 © Webmaster's Talks
Design by 7dana.com