Author Topic: Maousec Security Conclusions!  (Read 10072 times)

Offline pandlouk

  • I love Comodo
  • Comodo's Hero
  • *****
  • Posts: 2240
  • Retired Mod
Maousec Security Conclusions!
« on: December 28, 2006, 02:33:04 AM »
Here are the Security conclusions of Matousec for the tested firewalls


Kerio Personal Firewall 4.3
All the hopes died when it came to the analysis of the security and security design of Kerio. If we say that Kerio Personal Firewall implements no security at all we would not be far from the truth. The security design of Kerio is useless. The functionality of Kerio is not much better than the functionality of common packet filter. Except the packet filter none of its security components works as it should. Malware can bypass all the protection of Kerio easily. As you can see in the bug list below the number of bugs we found during our first phase analysis is not that big. It is not because there are missing many important features in the security design of Kerio and thus they can not be buggy. We have solved this unexpected situation by setting the unique penalty of these design bugs to very high values such that they are greater than possible sums of penalties of bugs in these features if they were implemented. The majority of bugs we found are critical bugs. The security of Kerio Personal Firewall was not tested against modern malware techniques if tested at all. We do not think there might be a worse personal firewall from the security point of view than Kerio Personal Firewall 4.3.246. We strongly recommend all its users to change the personal firewall.

BlackICE PC Protection 3.6
We have already mentioned above the basic security concept of BlackICE - the baseline. Unfortunately, this is the only idea of its security design. Moreover, its implementation is very naive and thus it can be bypassed very easily by malware. The situation of the security of BlackICE PC Protection is even worse because even in the concept with only one simple idea there are plenty of bugs in its implementation. Developers of BlackICE also forgot to implement crucial features for security of personal firewall. These missing crucial features are listed as design bugs in the bug list below with very high unique penalties such that their values are greater than possible sums of penalties of bugs in these features if they were implemented in BlackICE. Another security flaw was already mentioned, BlackICE does not support protection of its settings. The overall bad impression of the security of BlackICE is underlined by the fact that the whole application is distributed in debug mode. The security level and the software quality of BlackICE PC Protection 3.6 is the lowest possible and we can not recommend this product to anyone.

ZoneAlarm Pro 6.1
The most important thing on the security product is the security. This is the part of the analysis in which we pay extreme attention to all details to find every possible security vulnerability. As we can say ZoneAlarm Pro is outstanding or at least better-than-average in all other parts we can also say that this product totally failed during this part of our analysis. The first phase security testing of ZoneAlarm Pro resulted in unexpected amount of bugs. We have found over 30 bugs in which about 15 are critical. This would be simply too much even for the extensive analysis of ZoneAlarm's source codes but we have found so many bugs in the first phase of our analysis testing only a limited set of situations and features with our own methods and tools without any source code of ZoneAlarm Pro leaving the most common personal firewall problems to next phases. It looks like ZoneAlarm Pro was not betatested at all for the security issues. Its programmers lack important knowledge needed for writing security products for Windows NT operating systems. Our conclusion is that the level of security protection offered by ZoneAlarm Pro is insufficient for any kind of utilization.

Norton Personal Firewall 2006
As we have mentioned above Norton Personal Firewall 2006 is very huge software. This affected the security of this product too. There are many componenets and programs that are not connected well from the security point of view. The whole security is half-implementated and there is number of security holes even though there are a few components with quite well implemented security. But as a whole product Norton Personal Firewall does not meet the security level for any kind of use, not even for common home users. We have found several critical bugs and we have also found that many important security features miss completely. It is very probable that deeper analysis would find much more bugs. Norton Personal Firewall 2006 was not betatested properly and its programers miss basic knowledge of programming Windows NT security products. Professional security products must be implemented by security experts not only by programmers of application software. Based on the result of our analysis we do not recommend using Norton Personal Firewall 2006 at all because its security is very poor.

Outpost Pro 4.0
The security design of Outpost 4 is quite good but it still have major holes. Its vendor put stress on Anti-Leak protection that we do not test in this phase of our project. However, we have found many vulnerabilities that can be exploited by attackers to easily bypass this Anti-Leak protection as well as all other security mechanisms in Outpost. Not only the design but also its implementation is imperfect in Outpost. We have found components of Outpost that are more buggy than working. All these results in a very unstable application that is likely to have compatibility problems with common security software. Because of this, we can not recommend using Outpost. Vendors of widely used security products should have security level betatesters not only testers on the application level. It is clear that the development of Outpost missed this kind of testing.

Comodo Personal Firewall
The simplicity of the whole product is also visible on its security design. Unfortunately, this simplicity also means that some important security features were not implemented at all. Apart from various security design features, we have also missed the possibility to protect the configuration settings of Comodo Firewall using password, which is very common in competitive products. This can be a problem on computers that are used by more users at once. The positive on the security of Comodo Firewall is its excellent ability to fight against leak-tests. It probably was a priority of its vendor to pass all leak-tests. Only the Coat test was able to bypass its protection but we have been informed that the next version of Comodo Firewall will handle this one too. The implementation of the security design is very superficial. Today's malware creators would not have problems to bypass the protection of Comodo. The development of this firewall probably missed independent betatesting of its security features because the number and the nature of bugs we have found in it is alarming. This is why we can not recommend Comodo Personal Firewall as a personal firewall solution to anyone who require the real protection against today's malware. You can see the public information about bugs we found in Comodo Personal Firewall in the following sections below.
« Last Edit: December 28, 2006, 02:42:52 AM by pandlouk »

Offline pandlouk

  • I love Comodo
  • Comodo's Hero
  • *****
  • Posts: 2240
  • Retired Mod
Re: Maousec Security Conclusions!
« Reply #1 on: December 28, 2006, 02:35:33 AM »
So which firewall should we use? ???

None! just do not connect in internet. :P ;D

Offline AOwL

  • Comodo SuperHero
  • Comodo's Hero
  • *****
  • Posts: 2349
  • Comodo Firewall Pro - Be safe, use protection...
    • NordicNatureMedia
Re: Maousec Security Conclusions!
« Reply #2 on: December 28, 2006, 04:17:18 AM »
I wonder what firewall the Matousec guys are using... ???
I seems that none is good enough for them... ::)

This sounds a bit worrying though...
"Today's malware creators would not have problems to bypass the protection of Comodo."

Offline Toxteth O'Grady

  • Comodo's Hero
  • *****
  • Posts: 593
Re: Maousec Security Conclusions!
« Reply #3 on: December 28, 2006, 04:29:01 AM »
I wonder what firewall the Matousec guys are using... ???
I seems that none is good enough for them... ::)

This sounds a bit worrying though...
"Today's malware creators would not have problems to bypass the protection of Comodo."


That does sound worrying, doesn't it? But I wonder what it means. It could be anything. Shut down CPF,  connect to the internet when CPF is running or modify it in some way. And probably there are more possibilities...
Hopefully the developers know what these weaknesses are.

BTW, I think I read somewhere that Matousec (or whatever he is called) doesn't use a firewall.

Offline elfstone

  • Newbie
  • *
  • Posts: 17
Re: Maousec Security Conclusions!
« Reply #4 on: December 28, 2006, 04:32:43 AM »
Well... no leak-proof security product will ever beat the biggest liability of a computing system: the user :)

I am a bit annoyed by the fact that everything Matousec puts out is intentionally fuzzy and gloomy and disturbing. And poorly written. To paraphrase - reviews should be written by technical writers not only by security experts or whatnot.

Offline Toxteth O'Grady

  • Comodo's Hero
  • *****
  • Posts: 593
Re: Maousec Security Conclusions!
« Reply #5 on: December 28, 2006, 06:24:56 AM »
It's probably on purpose. Because he wants to make money from the bugs and flaws he finds, there is not much sense in being too specific. Perhaps the idea is by making it sound like the tested firewall is "worthless", the worried users will more or less force the producer to buy his reports.

Offline TheTOM_SK

  • Comodo Loves me
  • ****
  • Posts: 121
Re: Maousec Security Conclusions!
« Reply #6 on: December 28, 2006, 08:55:04 AM »
BTW, I think I read somewhere that Matousec doesn't use a firewall.
Security experts can set up PC, so it does not need a firewall, but about 99,99% people, including me, do not belong to this group, so we need a firewall. I wonder then, to whom does Matousek write those reviews for? The same can be said about AV, none will catch 100%, so because of that, we should not use any AV, sounds pretty silly, does not it? There is not a perfect firewall, bugs free and with 0 MB RAM usage, but at least Comodo belongs to the top and it still improving and taking user sugestions into account and that is awesome.

Offline panic

  • Global Moderator
  • Comodo's Hero
  • *****
  • Posts: 11825
  • Linux is free only if your time is worthless.;-)
Re: Maousec Security Conclusions!
« Reply #7 on: December 29, 2006, 09:41:54 PM »
Security experts can set up PC, so it does not need a firewall, but about 99,99% people, including me, do not belong to this group, so we need a firewall. I wonder then, to whom does Matousek write those reviews for? The same can be said about AV, none will catch 100%, so because of that, we should not use any AV, sounds pretty silly, does not it? There is not a perfect firewall, bugs free and with 0 MB RAM usage, but at least Comodo belongs to the top and it still improving and taking user sugestions into account and that is awesome.

When you're finished hitting the nail on the head, can I have my hammer back?

I'm sure Matousec's testing process was rigourous and his results are, quite probably, correct, in a technical sense. I say this despite them reporting BSOD's and "denial of service" being listed as minor bugs and the malware exploits that they have exposed as being the major ones. I"m equally sure that the Comodo developers are only too well aware of the results and are working on it even as we speak.

Given that Matousec say that they can't recommend any firewall for Windows, is the logical alternative to use nothing?

With all due respect to their technnical expertise, I fail to see how this could possibly be the recommendation of a security company.

Excuse me while I just pull the plug to improve my securi
As your mums would say, "If you can't play nice with all the other kiddies, go home".
All users are asked to please read and abide by the  Comodo Forum Policy.
If you can't conform, don't use the forum.

Offline elfstone

  • Newbie
  • *
  • Posts: 17
Re: Maousec Security Conclusions!
« Reply #8 on: December 31, 2006, 03:53:06 AM »
Now, really... they don't recommend that you use no firewall. They just say they can't recommend any particular firewall.

I think this is a good thing, even. It's eventually up to you to decide.

It's safe for them at any rate. Imagine they recommended a firewall, and the next day some guy exposes a big mega-hole in that firewall.


Offline munckman

  • Comodo Family Member
  • ***
  • Posts: 89
Re: Maousec Security Conclusions!
« Reply #9 on: January 01, 2007, 05:22:48 AM »
Now, really... they don't recommend that you use no firewall. They just say they can't recommend any particular firewall.

I think this is a good thing, even. It's eventually up to you to decide.

It's safe for them at any rate. Imagine they recommended a firewall, and the next day some guy exposes a big mega-hole in that firewall.


I have to agree with everything you say.

Offline egemen

  • Comodo Staff
  • Comodo's Hero
  • *****
  • Posts: 3380
Re: Maousec Security Conclusions!
« Reply #10 on: January 03, 2007, 09:00:44 AM »
I have to agree with everything you say.

Until they find a better one, they will recommend the best in their list which is CPF. Otherwise, it would be quite contradictory :)

Dont worry about it guys. CPF 3.0 is coming as a standalone antimalware solution. Until that time, CPF 2.4 is the best firewall that should be used with an anti-malware solution to provide maximum security.


Offline AOwL

  • Comodo SuperHero
  • Comodo's Hero
  • *****
  • Posts: 2349
  • Comodo Firewall Pro - Be safe, use protection...
    • NordicNatureMedia
Re: Maousec Security Conclusions!
« Reply #11 on: January 03, 2007, 10:29:19 AM »
I think Matousec is cowards that just play safe when they say that they don't recommend anyone of the firewalls.
I think that they have a responsibility to tell their readers which one is the best to use at the moment. What's the point with their tests otherwise?
Just because they recommend one, doesn't mean they think that it's 100% safe or without bugs. The chickens at Matousec can put that in a disclaimer if they are scared...

I totally disagree with Elfstone in his post above as you can see.... ;D

Offline egemen

  • Comodo Staff
  • Comodo's Hero
  • *****
  • Posts: 3380
Re: Maousec Security Conclusions!
« Reply #12 on: January 03, 2007, 11:00:30 AM »
I think Matousec is cowards that just play safe when they say that they don't recommend anyone of the firewalls.
I think that they have a responsibility to tell their readers which one is the best to use at the moment. What's the point with their tests otherwise?
Just because they recommend one, doesn't mean they think that it's 100% safe or without bugs. The chickens at Matousec can put that in a disclaimer if they are scared...

I totally disagree with Elfstone in his post above as you can see.... ;D

Go easy on them guys. AFAIK they are just talented and young university students. But i do recall somewhere in their site, they recommended ZoneAlarm pro because it was the best they had tested. This was ofcourse before testing CFP. But i do agree that they must clearly write this on their sites so that everyone can read.

Offline Little Mac

  • Forum Volunteer
  • Global Moderator
  • Comodo's Hero
  • *****
  • Posts: 6303
  • The Colonel told me to.
Re: Maousec Security Conclusions!
« Reply #13 on: January 03, 2007, 11:19:54 AM »
My only thought revolves around what seems to be a contradiction by the Matousec conclusions.  To quote:

"Comodo Personal Firewall
The simplicity of the whole product is also visible on its security design. Unfortunately, this simplicity also means that some important security features were not implemented at all. Apart from various security design features, we have also missed the possibility to protect the configuration settings of Comodo Firewall using password, which is very common in competitive products. This can be a problem on computers that are used by more users at once. The positive on the security of Comodo Firewall is its excellent ability to fight against leak-tests. It probably was a priority of its vendor to pass all leak-tests. Only the Coat test was able to bypass its protection but we have been informed that the next version of Comodo Firewall will handle this one too. The implementation of the security design is very superficial. Today's malware creators would not have problems to bypass the protection of Comodo. The development of this firewall probably missed independent betatesting of its security features because the number and the nature of bugs we have found in it is alarming. This is why we can not recommend Comodo Personal Firewall as a personal firewall solution to anyone who require the real protection against today's malware."  (refer to bold blue text)

So which is it?  If it's top dog against leak tests, how is it that today's malware can easily bypass it?   ???

LM
These forums are focused on providing help and improvement for Comodo products.  Please treat other users with respect and make a positive contribution.  Thanks.
Forum Policy

Offline egemen

  • Comodo Staff
  • Comodo's Hero
  • *****
  • Posts: 3380
Re: Maousec Security Conclusions!
« Reply #14 on: January 03, 2007, 12:00:14 PM »
My only thought revolves around what seems to be a contradiction by the Matousec conclusions.  To quote:

"Comodo Personal Firewall
The simplicity of the whole product is also visible on its security design. Unfortunately, this simplicity also means that some important security features were not implemented at all. Apart from various security design features, we have also missed the possibility to protect the configuration settings of Comodo Firewall using password, which is very common in competitive products. This can be a problem on computers that are used by more users at once. The positive on the security of Comodo Firewall is its excellent ability to fight against leak-tests. It probably was a priority of its vendor to pass all leak-tests. Only the Coat test was able to bypass its protection but we have been informed that the next version of Comodo Firewall will handle this one too. The implementation of the security design is very superficial. Today's malware creators would not have problems to bypass the protection of Comodo. The development of this firewall probably missed independent betatesting of its security features because the number and the nature of bugs we have found in it is alarming. This is why we can not recommend Comodo Personal Firewall as a personal firewall solution to anyone who require the real protection against today's malware."  (refer to bold blue text)

So which is it?  If it's top dog against leak tests, how is it that today's malware can easily bypass it?   ???

LM

Protection against today's malware requires more than a firewall but a HIPS. Thats the mistake they do while designing the methodology. For example, there are many operations in their criteria that do not directly belong to a firewalls list of tasks: Privilege Escallation check or protection of system resources for example.

A firewall is interested in network traffic. If you want to increase the level of the control, you will monitor some critical activity such a DLL injection or memory injection. CPF is not an anti-malware solution.

A malware can load a system driver and this means it can run as a part of windows kernel. Now detecting this type of activity is not a direct duty of a firewall. But a HIPS must do such a check. Or installation of a keylogger or infection of an executable.

According to the feedback from our users, we concluded that they do expect an antimalware solution when they install a firewall. So we have designed the new generation software CPF 3.0 which is actually a fully networking capable HIPS.

Anyone who can perform a search in our forums will see that, CPF, even not intended for that purpose, captured a couple of unknown malware before connecting to the internet with a clear message of this is a malware behavior.

So what they should have said: "If you want a powerful firewall and have the maximum control on the network traffic, you can use CPF. If you want an antimalware solution, complement CPF with a hips or AV."


 

Free Endpoint Protection
Seo4Smf 2.0 © SmfMod.Com Smf Destek