CIS vs NIS challenge

I took the CIS vs Nis Challenge but I did it with a twist. Normally when people test they tst up todate software with an internet connection. This time what I wanted to do is to see how well these two work at the core of the product, by that I mean what happens if you have a little older signautres and no internet connection, it should still keep you safe, even if they don’t have a signature for the malware.

So what I did is install both CIS and NIS yesterday. I did a full update on both and then shut down the machines. From yesterday to today I found malware that at the time they has very low detection ratios and neither CIS or NIS had signatures for.

Today I did the test, check out the results.

Video 1 - YouTube

Video 2 Comodo Vs Norton 2011 Editions Video 2 - YouTube

Video 3 Comodo Vs. Norton 2011 Editions Video 3 - YouTube


This is the second 2, this time both products are updated and the internet is on. Both are ran against the same malware.

Video 1 Comodo vs Norton 2011 Editions Test 2 Video 1 - YouTube

Video 2 Comodo vs. Norton 2011 Editions Test 2 Video 2 - YouTube

This is an interesting test that shows day Zero capability!

Great review!


It seems that people don’t understand what this test is supposed to show.

I think I know what you are driving at. Often my laptop [my personal laptop] is unhitched from any network, yet I am running a lot of software. This means no Internet, no “cloud,” no security mothership to connect to. I need my security software to perform w/o calling for additional help. Before I found Comodo I used some other, very well known, security solutions under these conditions–they utterly failed me. Comodo has kept my laptop clean, on or off the Internet, on or off a network. For me a clean computer means I work faster. If I work faster I make more money. Simple. Norton failed me horribly leaving me in quite a spot. I have not forgotten. Made me just a little PO’ed. >:-D

Useless test IMO. I completely agree with the guy who posted this.

I don’t understand the point of having updated norton a day ago you should have updated it when you were about to do the test. Thats my opinion It could have had a chance to catch more not just relying on a sandbox that blocks all unknown applications. Sonar is a great module and being disconnected from the net is testing a limited norton.

If you’re going to test Norton with one of it’s key features disabled, then test Comodo without the sandbox. Then show the results.

So all Norton products that require 24/7 connection, should carry this warning on it’s product.

If you are not connected 24/7 to the internet this product may fail.

Sorry you are testing a limited Comodo no cloud.


Great videos!!! I see nothing wrong with these tests and just goes to show how well Defense+/Sandbox protects your system :).

Working in IT myself, I’ve seen so many users computers who’s antivirus/security suite has expired and hasn’t been renewed. If only CIS has been pre-installed on these systems.


Thanks for Languy99

It will open too many peoples eyes.


This test, imo, shows the “prevention” capability of the products. It shows what happens to an AV product if it doesn’t have “prevention” capability. This test shows if the AV is a “reactive” or “Proactive” security product.

If you rely on “detection” then you always need to update your signature DB and you can never ever guarantee you have 100% of the malware in your signature db. Which means you are bound to get infected.

This is why its important to build “prevention” architecture at the heart of your security product and not rely on “detection” to secure people.

This test shows this very clearly.



I haven’t watched the videos yet, but I think I know what this test is about.

There are different ways in which an infection may get onto your computer. One is downloaded from the internet. This test isn’t about that. Another way is through a flash drive. It sounds like this test may be about that sort of thing when you don’t always have an internet connection.

My point is that there are scenarios in which testing out of date signatures without an internet connection is valid. They don’t apply to everyone, but it is still an important test for a great many people.

thank you for showing us that paid security it not better than free. And you have shown that CIS is better NIS.


I find it funny that people are telling me to make it fair I should turn off sandbox and D+ in comodo. What I find more funny is that the only way to show comodo failing is to disable all of it’s technologies. People don’t understand that test was 100% fair ( same malware, same testing conditions) If I tested another product I bet it would not have failed, something like online armor. What is not fair is symantec telling you, you are safe ( but forgetting to tell you that you need an active internet connection to do it). What symantec needs to do is to develop something that will work no matter what, something that does not rely on the internet or signatures. Without that all you have is a glorified detection engine.

Norton Antivirus 2011 - updated signatures (malware samples are 3 days old)

My small prevention test…

by disconnecting from internet, you also disabled Comodo’s Behaviour Analysis, Cloud AV and Cloud Whitelisting…


So what was the outcome here? Did Norton protect against this malware?



Hey Melih, go to the videos and read the comments. Tell me if you see anything interesting. :wink:

Norton failed to protect the computer… :-TD

I’m trying to figure this out. What counts as an infection? Is there a definition?

My way of looking at this is whether malware can harm the computer in any way or steal information, but malicious files sitting on the computer don’t count as an infection. Do they have to be active?

Does anyone have a good definition? For this review I didn’t see the point of looking at how much malware Kaspersky found because it was obvious the computer was dead. Why did it matter if it took one sample or five?

An infection is an infection, right?

Also I just read through the comments for the videos.

Why are so many people saying that the results wouldn’t have been the same if the sandbox hadn’t been there? ???

Isn’t this the same sort of thing as in an ordinary test saying that the results for Norton wouldn’t have been the same if the signatures hadn’t been there? What’s the difference?

Also, little do they know that if you disable the sandbox you’re still completely protected by Defense+. You’ll just get more alerts.