New Matousec Firewall Challenge

[Guest]

[ailef]

the joke of the year : kaspersky zonealarm and online armor are better than comodo....
yes i know that and my mother is 100 meters world champion, didnt u know this info maybe ?

[Melih]

Egemen has informed David (Matousec).

We hope he will correct it soon. We also hope as a goodwill gesture he will test the bug fixed version and update his results accordingly.

This shows that its important not to rely on a single person's test results and use a full test application like testmypcsecurity.com in order to avoid the human error.

Thanks

Melih

[Star Shadow]

Actually, it would be a good gesture if they also tested the free version of OA to give it accurate results. Not too many people are going to read the fine print that says that it was tested with the old tests. I think the free version of OA will perform less well than Comodo's Firewall. So, tell David to also update the that test as well to be "fair". Fair is fair after all.

[HarmonicShadow]

I am just absolutely shocked by how far CIS fell from CFP 3.0's rating.  CFP was the 3rd highest on the list, last time I checked and the highest FREE firewall on the entire list.  Now, it appears that CIS has fallen farther than CFP.  The SSS shutdown stuff is what halted CFP and that probably was resolved the update afterward, though Matousec never tested CFP after that update to see if it was fixed.  Now that CIS has come out, it appears those vulnerabilities have still not been resolved or resolved only partially.  Defense + should be updated to intercept any Shutdown, Log Out, Restart, Standby, and Hibernate request by an unknown application.  That would put a red flag to someone that if a program is about to restart the machine and they don't know what the program is, they can stop the restart.  As for other tests, CIS needs some holes fixed.  Some people are probably looking at that and feeling much safer with PC Tools Firewall or Netchina.  I know they added more tests to challenge them, but it seems as though CIS is still having trouble with old tests.  Nothing is perfect, but Comodo needs to step it up or they might end up losing that Very Good level of protection.  We already lost excellent.  I'm afraid to say this, but I believe that Online Armor Pro will be the first firewall ever to score a perfect 100% on the firewall challenge.  Why I say this?  Because as soon as Comodo updates CIS to fix these problems, it should be immediately retested by Matousec.

[Leolas]

I'll send him an email too, about the OA thing. It has no sense.

[Melih]

I am just absolutely shocked by how far CIS fell from CFP 3.0's rating.  CFP was the 3rd highest on the list, last time I checked and the highest FREE firewall on the entire list.  Now, it appears that CIS has fallen farther than CFP.  The SSS shutdown stuff is what halted CFP and that probably was resolved the update afterward, though Matousec never tested CFP after that update to see if it was fixed.  Now that CIS has come out, it appears those vulnerabilities have still not been resolved or resolved only partially.  Defense + should be updated to intercept any Shutdown, Log Out, Restart, Standby, and Hibernate request by an unknown application.  That would put a red flag to someone that if a program is about to restart the machine and they don't know what the program is, they can stop the restart.  As for other tests, CIS needs some holes fixed.  Some people are probably looking at that and feeling much safer with PC Tools Firewall or Netchina.  I know they added more tests to challenge them, but it seems as though CIS is still having trouble with old tests.  Nothing is perfect, but Comodo needs to step it up or they might end up losing that Very Good level of protection.  We already lost excellent.  I'm afraid to say this, but I believe that Online Armor Pro will be the first firewall ever to score a perfect 100% on the firewall challenge.  Why I say this?  Because as soon as Comodo updates CIS to fix these problems, it should be immediately retested by Matousec.

Pls read Egemen's response above...It explains the error by Matousec.

Melih

[Kyle]

Thank you for the response Egemen and Melih

[hullboy]

COMODO should also ask Matousec to stop testing version 2.4.18.184 that is no longer developed (L)

[DarkButterfly]

Probably someone already mentioned the following, but doesn't the testing look so, and I at this moment I can't find a better word as I just woke up, but, doesn't it look like a total crap? I mean, testing simple firewalls against firewalls+hips against behavior blockers. Would this be like saying that, by nature, a bigger tomato will have more juice? Odd analogy? Maybe so...

And who cares if CIS lost some ground. I'm sure, and as Melih and Egemen already stated, next tuesday a new version will come out that will fix this. As Melih also said, I truly hope that David will be willing to retest CIS. Or will be this a matter of money? I think people deserve, specially the new people, to know that CIS (new version) is able to block those tests already.

Of course, such tests, reveal what needs to be improved in such security tools. That's great. I wonder, though, if he shouldn't first get in touch with security vendors in order to fix those issues, before he places the results on-line. I'm not an expert, but couldn't this be a little dangerous, for the bad guys, to know such vulnerabilities?

I say all this, 'cos I don't think he sould just be providing independent testing, but also a security service, and by firsy placing the results on-line,  I don't think he is. Maybe I see this the wrong way. Then again, I'm not that an expert.

[hullboy]

I think that Matousec did honestly the job that he is meant to do... that is "testing firewalls"

Matousec tests firewalls -> some bugs come out -> the "firewall producer" fixes them -> the "firewall producer" informs Matousec -> Matousec retests the firewall (with the chosen schedule)

I expect that, since Matousec runs his tests with a certain schedule, it is difficult to obtain that some days after that he published the results he should re-run all tests just because a producer says that he fixed the bugs, because it would become a daily "contest"

[lordraiden]

First of all i want to say that thanks to matousec all win.
Thanks to him comodo has discover today some bugs and they are going to release an update the next week to get 100% score.

Other apps like like outpost firewall, OA, ZoneAlarm,... take this test for improve their software, thanks to this the final user wins.

For all of this, I want to give thanks to Matousec for his work.

There is a poll in his web about firewalls, hips... and he for testing always choose the products in the firts places, mamutu was an exception because some user send an email to him.

It's normal that he want to earn money with this, "nobody" works for free and his test are the most profesional that we can find on internet, if somebody can do better i hope that he do.

Please stop to criticize and send an email to him with your opinions.
http://www.matousec.com/matousec/contact-us.php

[nick55555]

I wanna say a big THANKS to comodo developers for keeping us informed (L) I'm waiting for the new version... 

[Leolas]

I'll send him an email too, about the OA thing. It has no sense.

This is the response of Matousec team:

Hello Leo,

Thank you for your message.
We absolutely agree with you.
However, testing of one product takes 4-12 hours.
We have tested 8 product during last testing.
There was no time for testing more. When we test
more products together, we risk that a vendor of one of
the tested product releases a new version after we tested
the old version but before we published the results.
This is why it is no possible to test more than 8 products
during one update. The whole testing took 7 days.

Online Armor 3 Free will be tested during the next update
as well as others. But there is no way to test all products we are aware of
during a single update when the new tests are implemented.
Thank you for understanding.

Kind Regards,

--
Matousec - Transparent security Support
http://www.matousec.com/

[Kyle]

That's not right, He should reserve the updates and release them all at once.

When we test
more products together, we risk that a vendor of one of
the tested product releases a new version after we tested
the old version but before we published the results.
This is why it is no possible to test more than 8 products
during one update. The whole testing took 7 days.

If he cannot publish equal testing at the same time then perhaps he should get more people to work for him or Volunteer's.

It's missleading to the general public when they aren't tested at the latest versions\new tests, they could score better or worse.

[Star Shadow]

It's missleading to the general public when they aren't tested at the latest versions\new tests, they could score better or worse.

I agree. It is very misleading. Like OA Free could do extremely bad with the new tests, just like OA Paid was moved down a few notches. Most people will not read the fine print and take OA Free to be really good when it could end up being really bad with the new tests. That page should have new tests only listed on it and previous tests on a different page.

[Tags:]