Public Usability Test (First Of, CIS v3.9 Beta vs Online Armor 3.5)

PUBLIC TEST OPEN!!!

Hey Everyone,

I recently conduced 2 tests, A pop up test comparing the number of pop ups between Comodo Internet Security v3.9.74199.494 and Online Armor Personal v3.5.0.9. Entire Report is below. In these particular tests, Online Armor had more pop ups then Comodo Internet Security did.

I would appreciate it if people contribute to this, Do there own tests so I can update the Excel Sheet. You can also test other products vs CIS if you wish. We are ideally testing v3.9 of CIS. So go ahead and share your thoughts and ideas! I am open to any suggestions or comments, But remember… Forum Policy Still Applies!!!

For anyone is not registered, or can’t open Excel here are the basic details:


Test 1 - Batch 1 - (OA VS CIS.zip attached)

Product Name And Version - 10 Applications Tested! Windows XP SP3 32bit. Production Machine.

SmartFTP v3.0.1026.32
Trillian v3.1.12.0
FlashGet v1.9.6.1073
FastStone Image Viewer v3.7
DVD Shrink v3.2.0.15
Alt-Tab Thingy v3.4.1
Free Download Manager v3.0.848
Orbit Downloader v2.8.8
Advanced SystemCare Free v3.2.0
VersionTracker Pro v4.1

Format Used:
-Install
-Launch
-Update

Total Pop Ups CIS: 8
Total Pop Ups OA: 20



Test 2 - Batch 1 - (OA VS CIS 2.zip attached)

Product Name And Version - 9 Applications Tested! Windows XP Professional Edition SP3 32bit. VMWare WorkStation.

Download Accelerator Plus v9.1.1.1
GOM Media Player v2.1.17.4710
Camfrog Video Chat v5.2
FileZilla v3.2.3.1
LimeWire Basic v5.1.2
Winamp Full v5.551
MagicISO v5.5.274
SpywareBlaster v4.2
mIRC v6.35

Format Used:
-Install
-Launch
-Update

Total Pop Ups CIS: 19
Total Pop Ups OA: 22


Notes

  • Both Comodo Internet Security and Online Armor were tested at Default Settings. Not even Alert Options were touched! It was simply “Allow” and move forward.

  • For Test 1, Testing environment was in Windows XP Home Edition SP3 32bit. This was my very own production machine I tested on. I used Revo Uninstaller to uninstall applications in between the testing of Comodo and Online Armor.
    In Test 2, I fired up VMWare Workstation that was based on Windows XP Professional Edition SP3 32bit, Took a snapshot of it at clean state, and began the tests. After testing each application, All I had to do was restore to the snapshot and continue testing the products.

  • 10 Applications were tested, Detailed in the Excel Document attached. Entire Report is also included.

  • Testing was based on 3 things: Install, Launch, Update. This was the testing format used.

Cheers,
Josh

[attachment deleted by admin]

Excellent work Josh…

CIS ver 3.9 is the version designed to be used by Novices with default configuration! Please feel free to contribute by increasing the no of applications we can test against.

Thank you again Josh good work!

PS: we are working on the rest to reduce these alerts almost to zero!.. thats ver4!!!

Melih

I would appreciate it if people contribute to this
Maybe someone can do a test for kaspersky's next edition kis2010 and compare us to the competition (I know we win in the firewall department :)

I save people the trouble and give it a link, Here’s the link for it
ftp://downloads1.kaspersky-labs.com/beta/kavwks8.0_for_windows7/
P.S. kavwks8.0_for_windows7 <—I doesn’t have to be windows 7, but that’s there current built for it so far

I think you gave the wrong link. I think this is the last KIS 2010 link:

ftp://downloads1.kaspersky-labs.com/devbuilds/9.0.0.313/

geko

I uploaded it here (so anyone can see it, with or without Excel):
http://www.editgrid.com/user/jeremysbost/OA_vs_CIS

:a0

:stuck_out_tongue:

So these are like trusted apps, and the goal is to have them allowed without as much pop-ups as possible?

i see a bug in that. With Faststone capture, I had two popups. One to start, the second when I captured an image (it wanted to run the default image viewer software).

Yes. Legitimate Applications, And yep that’s the goal - By total default settings. This is PURELY a usability test.

These tests will prove very useful to novices and give a good indication where COMODO is at. So far, So good. :-TU So we would appreciate if people can add other applications to vs CIS v3.9.

Thanks Guys.

Cheers,
Josh

I have one question.

Is Comodo Internet Security 3.9.74199.494 in its default settings so strong as Online Armor Personal 3.5.0.9?

CIS has Buffer Overflow protection which accounted for around 60% of the vulnerabilities in 2008. Confiker was exploiting Buffer overflow vulnerability. So I will let you be the judge.

As I always said, we built one of, if not, world’s best security first. And as I always promised we are delivering the Usability part! So you have an unparalled security with ever improving usability and now this amazing security is usable by a novice.

Melih

And you are doing very well, Melih. Your team is creating a really great product.

how did you so the test? OA doesn’t open 20 popups, but 9.

btw, I could do something similar with other programs, and the reasult may be much different :-\

Lealos.

Being an Online Armor fanboy and moderator on there forums you are, If you are here to troll, Then I strongly invite you not to. This was an independent test made by my self, And ANYONE wanting to contribute may do so. And off course you can try other programs, and results may be different. This is again, An independent test and the first batch TESTED based on the Format used above.

Cheers,
Josh

First of all, this aint gonna be a OA is better, NO comodo, NO OA, NNOOOOOOO Comodo, NOOOOOOO (mcafee :P)etc etc

Long time no seen Leolas, how are you ?
Can I ask you if you were using a fresh install of OA ? Perhaps there were some rules you already had ?$

Xan

That’s not the only way for Conficker to attack. :wink:

By the way, why was this done against Online Armor, and not include other similar products, like Outpost and others?

But it’s an important part no-one else protects you against. the rest is blocked anyway by Defense+…

Xan

This is just a starter, I hope more and more tests will come up. but we can’t expect that members are going to uninstall and reinstall several products just to make a comparison :), but if you want to, please do so

Xan

As Xan said, This is only a first test. And I said at first, I do welcome anyone to contribute so we can update the Excel Sheet ASAP - Even with other products.

Cheers,
Josh

No, because of creating executables pop ups and so on it’s not representative in any way…

What a poor show :-TD

I don’t think that’s a fair comment there.

First, when someone starts something to rate a products vs other product(s), then its a this vs that. I don’t see it in a different way.

Then, you imply that Leolas had predefined rules. Can’t we say the same for Exist about CIS? (This sort of comment leads to other, etc… Leads us nowhere.)

Why don’t you guys just make a video showing the settings, do the test and upload it to youtube/similar service and no one doubts it? That would be the best way. Otherwise, only mirrors and smoke.