Are leak-tests really that important????

The main reason why I am asking this is because the latest review of Norton Internet Security 2007 in PC Magazine-which has very poor anti-leak protection was actually tested against the real malware and it blocked all of the real malwares for outbound connections.
And yet,Norton 2007 has a very poor anti-leak protection on Matousec’s leak-tests.

Can anyone please read this-it seems to me that leak-tests are un-important,look what Sunbelt Software - the vendor of Sunbelt Kerio Personal Firewall has responded to Matousec’s testings:

Sunbelt Software is committed to providing the strongest possible security products to its customers, and we will be working to correct demonstrable issues in the Sunbelt Personal Firewall. Users can expect these and other continuing enhancements for the Sunbelt Personal Firewall in the near future.

However, we have some reservations about personal firewall “leak testing” in general. While we appreciate and support the unique value of independent security testing, we are admittedly skeptical as to just how meaningful these leak tests really are, especially as they reflect real-world environments.

The key assumption of “leak testing” – namely, that it is somehow useful to measure the outbound protection provided by personal firewalls in cases where malware has already executed on the test box – strikes us as a questionable basis on which to build a security assessment. Today’s malware is so malicious and cleverly designed that it is often safest to regard PCs as so thoroughly compromised that nothing on the box can be trusted once the malware executes. In short, “leak testing” starts after the game is already lost, as the malware has already gotten past the inbound firewall protection.

Moreover, “leak testing” is predicated on the further assumption that personal firewalls should warn users about outbound connections even when the involved code components are not demonstrably malicious or suspicious (as is the case with the simulator programs used for “leak testing”). In fact, this kind of program design risks pop-up fatigue in users, effectively lowering the overall security of the system – the reason developers are increasingly shunning this design for security applications.

Finally, leak testing typically relies on simulator programs, the use of which is widely discredited among respected anti-malware researchers – and for good reason. Simulators simply cannot approximate the actual behavior of real malware in real world conditions. Furthermore, when simulators are used for anti-malware testing, the testing process is almost unavoidably tailored to fit the limitations of simulator instead of the complexity of real world conditions. What gets lost is a sense for how the tested products actually perform against live, kicking malware that exhibits behavior too complex to be captured in narrowly designed simulators.
Symantec Corp. - the vendor of Norton Personal Firewall.

Your opinions,needed,please!!!

Thanks a lot!

And of course I forgot to tell you that I personally think that inbound protection is much,much more important than outbound protection because,it’s crucial that malware doesn’t get inside your computer in the first place!
Detection and prevention are the most important for inbound protection!
Any other opinions???


Strong inbound prevention is the first point of any secured system however, the means and methods of getting junk inside your system are growing at a faster rate than security system can analyze and update.

There was a report issued recently (I THINK it was by Symantec, but am not 100% certain) that stated the ratio of legal software releases to malware releases, per-month, was growing past 1:80 (based on approx. 1000 legitimate release per month). This wave of malware will, inevitably, ouweigh inbound preventative methods, which is where outbound filtering and leak proofing become paramount.

Please understand that I’m not saying outbound and leak proofing are the “ducks nuts” of protection. A multi layered, multi directional protection strategy consisting of leak-proof bi-directional firewall, competent anti-virus, competent anti-malware/spyware and an ounce of knowledge is, IMHO, the way to go. What brands of these software layers you choose is your choice - it’s your system and you have to use software you feel you can trust.

Of course a hardware router on your perimeter with properly defined inbound protection is ideal on top of the software layers.

Hope this helps,
Ewen :slight_smile:

Hi,panic,I thanks for the answer,I thought this was an irrelevant question,and that’s why I was scared none will reply it.
That’s why I have to thank you for the answer.

Also, it is worth mentioning the comment about “pop-up fatigue”.

This can be a problem for some users with other firewalls and even the current CFP 2.4. However, the new CFP 3 is intended to minimize these pop-ups through use of a massive database of safe applications that will be continually updated. It is hoped that the new firewall, currently in beta testing, will be largely silent with regard to pop-ups and will provide a highly secure system without bothering users too much.


Well,despite everything,Panic,I truly hope that Comodo will be made to pass all of the leak-tests in the future?
Can Mehil,you and all of other Comodo’s techies promise that to ensure Comodo’s outbound protection?

Our Promise is to protect the Internet Population from many Threats that exist!
From Malware, to phishing to spam, Our promise is we will do everything possible to protect you!


Well if I had to say, outbound is not really a major component of a firewall. If you have resident scanners then outbound should not be held as heavily as inbound protection. For example, lets say CPF had crummy outbound protection, but your getting the best inbound protection. Would you rather get amazing outbound protection for crummy inbound protection? I think not.

I’d rather have great protection in both directions, regardless of whether the protecton is provided bu hardware, software or a combination.

Ewen :slight_smile:

Thanks a lot,Melih(and others as well).
I have only 3 short questions:
1.How would Comodo 3.0 fight against websites which contain all kinds of malware,how does the current version 2.4 fight against them?

2.When we can expect Comodo 3.0 Final-in the next 6 months period,possible?

3.Also,have you seen perhaps the HIPS test against kernel-level rootkits on the net,here they are:
termination tests for HIPS

some other tests for HIPS:

I know you’re extremely busy,I promise I won’t bother again.

1)V3 +CMG (until CMG is part of CFP, i must refer to it) won’t allow any malicious attemp by sites containing malware… they can contain what they like, but they aint going to make to a PC protected by CFP v3!

2)number of weeks rather than months. Latest Beta seems to be going in the right direction



I would prefer both I´ve had the outbound protection on 2.4 detect a a virus.
CFP reported my PC do a scan of PCs and tring to connect on port 2967. This virus got in as I had “Allowed all” for about 1 min this happened with symantec fully up to date Defs and all windows updates installed. I was however using an admin account at the time which I do not ussually do.

Without outbound protection I would have never known. (S) In order to be sure to wipe it I had to do a format/restore as it had installed a root kit on my pc and I never trust anything to fully remove those.

but also many virus can get in a pc/network after being invited by someone opening a bad email or even with a drive by download.

Anti virus does not catch everthing

Now I am starting trial implementation of a snort server, in hopes of installing one later in my office.

I have also implemented link logger on a linksys router as I was having bandwidth issues in the office. this shows my traffic as well as my major bandwidth users Mostly sendin lots of junk powerpoint & mpeg movies. I found out the users right away in the first day.

OD (J)

So if at an inbound scan all your ports come out stealthed but you fail an IMCP echo, is that really something to be concerned about?

IF your ports were stealthed, you wouldn’t fail an ICMP echo test. If you failed an ICMP echo test, then you aren’t stealthed.

How were you able to be stealthed but fail?

Ewen :slight_smile:

That wouldn’t be possible if you were stealthed since there would be nothing there to send the data to in order for it to echo reply. :-\

Hello, :slight_smile: first of all I’m no security expert but icmp doesn’t use ports.
some reading

icmp is blocked by code type, please see my firewall rule screenshot.

[attachment deleted by admin]

That’s quite correct it doesn’t,but the point is that in true stealth mode no response would show can disable ICMP redirects, to prevent a denial of service attack, by using Regedt32 to navigate to:


On the Edit menu, Add Value name EnableICMPRedirects, a type REG_DWORD entry, and set the data value to 0. A data value of 1 enables ICMP redirects.

hello, what average home user actually does this that you detail in your post?

IF your ports were stealthed, you wouldn't fail an ICMP echo test.

My reply is based on the context of this post.

It depends what you consider an average home user.Disabling unnecessary and insecure services is something that a great many security conscious users do.

EnableICMPRedirects may be true for W2K, but there is no s for that entry in XP.