Comparative Dynamic testing from NSSLABS

IE has done a LOT to improve security. But I would not considered it a securer browser. Its too targeted. Reps to NSSlabs if they tries to be AMTSO compilant.

I think its a totally gay that making “AMTSO” tests should take so much time… Are people idiots? Also testing organisations should share ALL samples so third parties can confirm testing results. How can we know those test organisations are just not trolling? Its sad, but there is a “very” low standard on all the Antivirus tests around… I only trust default DENY… (CIS D+) All AV’s sucks. :-TU :-TU

Its too targeted.

A secure targeted browser or an insecure less targeted one?
Hell of a choice.
I’d still go for IE8.

Well said. :-TU

:o WOW taking too much time testing a security product makes someone gay, WOW 88)

Well, we’ll have to agree to strongly disagree. Any study without information about how it was carried out is suspect, and especially often doesn’t necessarily mean what is promoted it meaning. I’ll just reference the many tests like AV comparatives that rank Comodo low - except there you can see why the study is flawed because they do tell you. But, for instance, the PC world rankings, or Symantecs claims which they do not back up, you all find suspect.

Finally, the whole anti-malware portion of that study was ridiculous - on par with the rankings of Comodo where the user clicked to allow all alerts. First, browsers are not Anti-malware products. Secondly Opera has been vulnerable to far less exploits for far shorter times for any sort of drive by malware attacks. The only test done by NSS (at least what became public) was in the anti-phishing tools. Which is great, but phishing isn’t malware by any definition I’ve ever heard.

I could go on, but I’ll just finish by indicating my surprise that any security conscious user would use IE of any version.

Well I don’t think that there is any evidence that IE is the safest. I high doubt that Smart screen filte (probably the main reason why IE scored so well when tested against opera) is as good today as it was back in beta. There was no reason for malware writers to test against it back in the days… Also those “socially engineered malware” thing that NSS Labs tested against don’t sounds like some real ■■■ threat.

Security flaws is to me more interesting than proving what sites/malwares a browser blacklist. And ofc what functionality it can offer.
I think Opera blog did a pretty solid bash of the study… If NSS has responded I would be happy to read the response…

Just from previous history I think its “probably” more likely to pop up a worm that exploit a hole in IE than in Firefox/Opera/chrome. Sure IE can show that it blocks this or that page but its highly unlikely that you will find phising sites that does NOT bypass IE8 the first few days/hours… Then when protection/blacklist is added the hacker start a new page… Game on…

The same with viruses, AV’s looks great. They catch nearly all viruses… But usually all the samples are months/week/days OLD… Old threats is no longer a valid one… Hackers has already moved on, and those who are infected got their AV disabled BEFORE any definitions was pushed out to the users. Back in the days IE was famous for downloading shit without an alert… Today that is not as common. Today user is more in charge… If I understood this test correctly it was a test to see what browsers blacklisted some suspicious sites and downloads from happening. While this might protect the “ordinary” dude to some extent its hardly nothing of interest to a semi advanced user, its an attack common only to the masses. NSS labs has not shown any real security flaws… Just tested the blacklist…

Those last three are not in the same league.
There’s IE; there’s Firefox…and there’s the rest.

And that’s why Opera always reacts so agressively: they never managed to gain market share.
Of course you could always say that users don’t know what’s good for them. But they’re the ones who decide in the end.

Come, come… I’m not unconscious.

Please tell me, are you contesting the result of the browsers’ tests or the methods NSSLABS used in order to reach such a result?

In any event, do you claim that Opera can block more malware than IE8?


I’m mostly arguing that the test was suspect as described, and that the test was also mostly meaningless. I also claim that neither Opera nor IE block malware as far as I can tell. If you want to argue that IE has a better phishing blacklist, I’ll concede that that is likely true, but also that anti-phishing isn’t at all the same thing as “malware blocking”, whatever that ought to mean for a Web browser.

I will claim that Opera is historically and currently less exploited than IE - though whether that is due to architecture or market share is unknown, and not at all proven by the NSS tests as far as I can tell.
See, for instance:
About Secunia Research | Flexera -Opera 10
About Secunia Research | Flexera -IE8

You can go back along the versions on that site and see the obvious disparity in vulnerabilities to actual drive by or any exploits.

Is there anyone who can actually explain how those “malware catch rate” and “Socially Engineered Malware Protection Over Time” results were computed and outline the underlying math? < >

Despite that metric is used to draw conclusions, for some reason is neglected by the methodological description.

Since Dynamic testing are starting to take place it would be interesting to unravel the mystery behind the “magic” formula whose results are highlighted in the summary of NSS report, especially accounting for the inevitable transformations that reverberate through media highlights or hearsays.

My previous questions still stand. IE 8 is more popular that Opera 10 therefore, subject to more attacks. IE 8 does block more malware than Opera 10 and I do not mean phishing attempts. I did my own un-scientific test with a bunch of malicious URLs and my own test result confirmed the findings of NSSLABS. IE 8 does block more malware “not phishing” than Opera 10.


Hi Melih

You asked me to show you where did NSSLABS state that they follow AMTSO guidelines and I did. Now I would like to know your opinion of their test. Is CIS going to participate in their testing in the near future?


Would you mind clarifying what you mean by “block more malware”? This is the problem with the NSS tests that we’ve been highlighting, but you own testing isn’t telling me anything about what you are talking about.

I can’t tell if you’re claiming that because IE8 is used more, so is numerically (not by percent of use, but just more people use it) attacked more often, so is blocking more malware because of numbers of people using it. If so, then you’d also claim that Symantec blocks more malware than Comodo because far more people use Symantec? That’s true, but certainly not what most people would interpret “blocks more malware” to mean.

Are you saying that you went to Malicious URLs with IE8 and Opera 10, and on some sites for each, your test PC was infected by drive by malware? And that IE8 was exploited less often than Opera 10?

I conducted a similar test like Matt’s:

Internet Explorer SmartScreen Filter Vs. Malware - YouTube ( IE8 )

Google Chrome Anti-Malware Test - YouTube (Google Chrome)

FireFox 3 AntiMalware Tests - YouTube (FireFox)

If you want I could re-do my test again, browsers that I could include are: IE 8, Opera 10, FireFox 3, Google Chrome and Safari 4. What do you think?


You really disappointed me by your answer :'(. There were more people using IE7 than any browser during the time of IE7 and yet IE7 did not block more malware than FireFox for example. Could you tell me why? ???

This thread was not created for the purpose of discussing the comparative browser test. The thread was created for the purpose of discussing the Consumer Endpoint Protection test from NSSLABS.

If one feels the need to discuss at length another subject, please create a specific thread for that particular subject.


I don’t want to go off topic, and I understand that. I will just close with saying that these sorts of tests seem to me to be pointless. Instead of alerting me to an attack that tried to exploit a vulnerability, how about just not have that vulnerability? I’ll leave this thread now with my main point that I don’t trust NSS labs or their testing.

From my own experience IE8 blocked and shut down a page trying to scan with and install the Personal Antivirus rogue before CIS even got a shot at it. If that’s not blocking malware, I don’t know what is. The SmartScreen filter is great. Personally, I found Opera’s claims against the browser test to be spurious at best and a futile attempt to discredit something that showed them in a bad light. I think that test was accurate for as far as the testing went, which I thought was pretty extensive and certainly included more than just phishing protection. Also, the fact that more people use IE contributes to the effectiveness of Smart Screen over what other browsers use. The more people reporting malware sites , either manually or automatically, the better the system works.

I think the browser tests NSS did are releveant to this thread since they would serve to increase or decrease people’s faith in any other other tests they do.

The better question here for me is, does Opera somehow download and install malware from that page that IE8 blocked? Or is the lack that Opera(Firefox, Chrome) didn’t bother to tell you that it was not running an IE exploit? I think I see the difference of opinion, apparently NSS and some posters here think the browser should be an Anti-Malware product and prevent you from explicitly downloading, saving and then executing an executable from the internet. I, and the Opera developers (and Firefox, Chrome) think that the browser should not be an AV product, nor should it prevent you from manually downloading something. After that, I figure that’s what the Anti-Malware product you trust (in this case Comodo) should be the one determining if the executable is malware.

To make this relevant to this thread, I feel like NSSLabs in the browser test tested products for something they don’t claim to do, and then said they did poorly at it, while implying they failed at a task they set for themselves. Then NSSLabs refused to let other people test the URLs that they used to corroborate or for other vendors to improve their products. This is irresponsible IMO - why not at least give the vendors who did poorly the ability to try and improve? What about responsible disclosure? All of that makes me not trust NSSLabs much.