Leak Test Results

Comodo Internet Security - CIS News / Announcements / Feedback - CIS
#1

Updated: 27 November 2008

Hi Everyone

In order to make sure everything is easily found, lets put the results of our test results to this thread. So pls go ahead and report back your test results here and lets have at least 2 or more people confirm the results before we can update our results. Then we can put these results at testmypcsecurity.com for everyone’s benefit.

Here are the product list i have got from testmypcsecurity. by all means if you are testing other products pls include it here.

These scores are out of 340. The highest that can be achieved is 340

Product Result
Comodo Internet Security --------------->340/340
Kaspersky Internet Security ------------->270/340
Agnitum Outpost Firewall Pro ------------>250/340
Jetico Personal Firewall
Sunbelt Personal Firewall --------------->50/340
Avira Premium Security Suite ------------>90/340
Online Armor Personal Firewall ----------->290-340/340
Online Armor Free ---------------------->290-340/340
Norton Internet Security 2009 ---------->50/340
BitDefender Internet Security ----------->20/340
ZoneAlarm Pro Firewall ------------------>220/340
ZoneAlarm Free Firewall ----------------->40/340
Iolo Personal Firewall
Panda Internet Security 2009 ------------>20/340
GoldTach Personal Firewall --------------->40/340
AVG AntiVirus Plus Firewall -------------->20/340
Spyware Terminator 2.5 ----------------->80/340
F-Secure Internet Security 2009 --------->20/340
DefenseWall ---------------------------->300/340

Thank you

Melih

PS: Mods: pls feel free to add the results to this post so that we have one place for all results.

CFW installation??
#2

Here is mine running CIS with proactive security:

COMODO Leaktests v.1.1.0.1
Date 09:51:02 - 16/11/2008
OS Windows XP SP3 build 2600

  1. Hijacking: ActiveDesktop Protected
  2. Hijacking: AppinitDlls Protected
  3. Hijacking: ChangeDebuggerPath Protected
  4. Hijacking: StartupPrograms Protected
  5. Hijacking: SupersedeServiceDll Protected
  6. Hijacking: UIHost Protected
  7. Hijacking: Userinit Protected
  8. Hijacking: WinlogonNotify Protected
  9. Impersonation: BITS Protected
  10. Impersonation: Coat Protected
  11. Impersonation: DDE Protected
  12. Impersonation: ExplorerAsParent Protected
  13. Impersonation: OLE automation Protected
  14. InfoSend: DNS Test Protected
  15. InfoSend: ICMP Test Protected
  16. Injection: AdvancedProcessTermination Protected
  17. Injection: APC dll injection Protected
  18. Injection: CreateRemoteThread Protected
  19. Injection: DupHandles Protected
  20. Injection: KnownDlls Protected
  21. Injection: ProcessInject Protected
  22. Injection: Services Protected
  23. Injection: SetThreadContext Protected
  24. Injection: SetWindowsHookEx Protected
  25. Injection: SetWinEventHook Protected
  26. Invasion: DebugControl Protected
  27. Invasion: FileDrop Protected
  28. Invasion: PhysicalMemory Protected
  29. Invasion: RawDisk Protected
  30. Invasion: Runner Protected
  31. RootkitInstallation: ChangeDrvPath Protected
  32. RootkitInstallation: DriverSupersede Protected
  33. RootkitInstallation: LoadAndCallImage Protected
  34. RootkitInstallation: MissingDriverLoad Protected
    Score 340/340
#3

COMODO Leaktests v.1.1.0.3
Date 21:31:38 - 15.11.2008

OS Windows XP SP3 build 2600

  1. RootkitInstallation: MissingDriverLoad Protected
  2. RootkitInstallation: LoadAndCallImage Protected
  3. RootkitInstallation: DriverSupersede Protected
  4. RootkitInstallation: ChangeDrvPath Protected
  5. Invasion: Runner Protected
  6. Invasion: RawDisk Protected
  7. Invasion: PhysicalMemory Protected
  8. Invasion: FileDrop Protected
  9. Invasion: DebugControl Protected
  10. Injection: SetWinEventHook Protected
  11. Injection: SetWindowsHookEx Protected
  12. Injection: SetThreadContext Protected
  13. Injection: Services Protected
  14. Injection: ProcessInject Protected
  15. Injection: KnownDlls Protected
  16. Injection: DupHandles Protected
  17. Injection: CreateRemoteThread Protected
  18. Injection: APC dll injection Protected
  19. Injection: AdvancedProcessTermination Protected
  20. InfoSend: ICMP Test Protected
  21. InfoSend: DNS Test Protected
  22. Impersonation: OLE automation Protected
  23. Impersonation: ExplorerAsParent Protected
  24. Impersonation: DDE Protected
  25. Impersonation: Coat Protected
  26. Impersonation: BITS Protected
  27. Hijacking: WinlogonNotify Protected
  28. Hijacking: Userinit Protected
  29. Hijacking: UIHost Protected
  30. Hijacking: SupersedeServiceDll Protected
  31. Hijacking: StartupPrograms Protected
  32. Hijacking: ChangeDebuggerPath Protected
  33. Hijacking: AppinitDlls Protected
  34. Hijacking: ActiveDesktop Protected
    Score 340/340

[attachment deleted by admin]

#4

Hi,

without testing guidelines all results a pretty meaningless.
Admin? LUA? Default program settings? Highest security settings?
You should really try to bring this to a more mature level, or it ends in another farce.

Cheers

#5

COMODO Leaktests v.1.1.0.3
Date 10:41:23 AM - 11/16/2008
OS Windows XP SP3 build 2600

  1. RootkitInstallation: MissingDriverLoad Protected
  2. RootkitInstallation: LoadAndCallImage Protected
  3. RootkitInstallation: DriverSupersede Protected
  4. RootkitInstallation: ChangeDrvPath Protected
  5. Invasion: Runner Protected
  6. Invasion: RawDisk Protected
  7. Invasion: PhysicalMemory Protected
  8. Invasion: FileDrop Protected
  9. Invasion: DebugControl Protected
  10. Injection: SetWinEventHook Protected
  11. Injection: SetWindowsHookEx Protected
  12. Injection: SetThreadContext Protected
  13. Injection: Services Protected
  14. Injection: ProcessInject Protected
  15. Injection: KnownDlls Protected
  16. Injection: DupHandles Protected
  17. Injection: CreateRemoteThread Protected
  18. Injection: APC dll injection Protected
  19. Injection: AdvancedProcessTermination Protected
  20. InfoSend: ICMP Test Protected
  21. InfoSend: DNS Test Protected
  22. Impersonation: OLE automation Protected
  23. Impersonation: ExplorerAsParent Protected
  24. Impersonation: DDE Protected
  25. Impersonation: Coat Protected
  26. Impersonation: BITS Protected
  27. Hijacking: WinlogonNotify Protected
  28. Hijacking: Userinit Protected
  29. Hijacking: UIHost Protected
  30. Hijacking: SupersedeServiceDll Protected
  31. Hijacking: StartupPrograms Protected
  32. Hijacking: ChangeDebuggerPath Protected
  33. Hijacking: AppinitDlls Protected
  34. Hijacking: ActiveDesktop Protected
    Score 340/340

Josh

[attachment deleted by admin]

#6

Win xp sp3 - CIS (firewall only with Proactive security) and Kaspersky AV 2009 - note: (Kaspersky blocks CLT dowloading and launch (:WIN) ) -

COMODO Leaktests v.1.1.0.3
Date 10:45:20 - 11/16/2008
OS Windows XP SP3 build 2600

  1. RootkitInstallation: MissingDriverLoad Protected
  2. RootkitInstallation: LoadAndCallImage Protected
  3. RootkitInstallation: DriverSupersede Protected
  4. RootkitInstallation: ChangeDrvPath Protected
  5. Invasion: Runner Protected
  6. Invasion: RawDisk Protected
  7. Invasion: PhysicalMemory Protected
  8. Invasion: FileDrop Protected
  9. Invasion: DebugControl Protected
  10. Injection: SetWinEventHook Protected
  11. Injection: SetWindowsHookEx Protected
  12. Injection: SetThreadContext Protected
  13. Injection: Services Protected
  14. Injection: ProcessInject Protected
  15. Injection: KnownDlls Protected
  16. Injection: DupHandles Protected
  17. Injection: CreateRemoteThread Protected
  18. Injection: APC dll injection Protected
  19. Injection: AdvancedProcessTermination Protected
  20. InfoSend: ICMP Test Protected
  21. InfoSend: DNS Test Protected
  22. Impersonation: OLE automation Protected
  23. Impersonation: ExplorerAsParent Protected
  24. Impersonation: DDE Protected
  25. Impersonation: Coat Protected
  26. Impersonation: BITS Protected
  27. Hijacking: WinlogonNotify Protected
  28. Hijacking: Userinit Protected
  29. Hijacking: UIHost Protected
  30. Hijacking: SupersedeServiceDll Protected
  31. Hijacking: StartupPrograms Protected
  32. Hijacking: ChangeDebuggerPath Protected
  33. Hijacking: AppinitDlls Protected
  34. Hijacking: ActiveDesktop Protected
    Score 340/340
    (C) COMODO 2008
#7

I think the issue will come when people have differnt scores for the same product, in which case I am sure the discussions between them will resolve the differences.

Its a good experiement about reallife scenerios and how products do in real life scenerios.

Melih

#8

Agnitum Outpost Firewall Pro 2009 (latest version) with HIPS component set for maximum protection and no automatic rules created.

COMODO Leaktests v.1.1.0.1
Date 23:15:55 - 16-11-2008

OS Windows Vista SP1 build 6001

  1. Hijacking: ActiveDesktop Vulnerable
  2. Hijacking: AppinitDlls Protected
  3. Hijacking: ChangeDebuggerPath Protected
  4. Hijacking: StartupPrograms Vulnerable
  5. Hijacking: SupersedeServiceDll Vulnerable
  6. Hijacking: UIHost Protected
  7. Hijacking: Userinit Protected
  8. Hijacking: WinlogonNotify Protected
  9. Impersonation: BITS Protected
  10. Impersonation: Coat Protected
  11. Impersonation: DDE Protected
  12. Impersonation: ExplorerAsParent Protected
  13. Impersonation: OLE automation Protected
  14. InfoSend: DNS Test Protected
  15. InfoSend: ICMP Test Protected
  16. Injection: AdvancedProcessTermination Vulnerable
  17. Injection: APC dll injection Protected
  18. Injection: CreateRemoteThread Protected
  19. Injection: DupHandles Vulnerable
  20. Injection: KnownDlls Vulnerable
  21. Injection: ProcessInject Protected
  22. Injection: Services Protected
  23. Injection: SetThreadContext Protected
  24. Injection: SetWindowsHookEx Protected
  25. Injection: SetWinEventHook Protected
  26. Invasion: DebugControl Protected
  27. Invasion: FileDrop Vulnerable
  28. Invasion: PhysicalMemory Protected
  29. Invasion: RawDisk Vulnerable
  30. Invasion: Runner Protected
  31. RootkitInstallation: ChangeDrvPath Vulnerable
  32. RootkitInstallation: DriverSupersede Protected
  33. RootkitInstallation: LoadAndCallImage Protected
  34. RootkitInstallation: MissingDriverLoad Protected
    Score 250/340

(C) COMODO 2008

Damn! I do love Comodo Firewall Pro (L)

#9

+1

Can you please provide a list of what should be included in the guidelines?

Ewen :slight_smile:

#10

CIS, using all Defense+ monitor settings on:

COMODO Leaktests v.1.1.0.1
Date 下午 06:17:27 - 2008/11/17
OS Windows XP SP3 build 2600

  1. Hijacking: ActiveDesktop Protected
  2. Hijacking: AppinitDlls Protected
  3. Hijacking: ChangeDebuggerPath Protected
  4. Hijacking: StartupPrograms Protected
  5. Hijacking: SupersedeServiceDll Protected
  6. Hijacking: UIHost Protected
  7. Hijacking: Userinit Protected
  8. Hijacking: WinlogonNotify Protected
  9. Impersonation: BITS Protected
  10. Impersonation: Coat Protected
  11. Impersonation: DDE Protected
  12. Impersonation: ExplorerAsParent Protected
  13. Impersonation: OLE automation Protected
  14. InfoSend: DNS Test Protected
  15. InfoSend: ICMP Test Protected
  16. Injection: AdvancedProcessTermination Protected
  17. Injection: APC dll injection Protected
  18. Injection: CreateRemoteThread Protected
  19. Injection: DupHandles Protected
  20. Injection: KnownDlls Protected
  21. Injection: ProcessInject Protected
  22. Injection: Services Protected
  23. Injection: SetThreadContext Protected
  24. Injection: SetWindowsHookEx Protected
  25. Injection: SetWinEventHook Protected
  26. Invasion: DebugControl Protected
  27. Invasion: FileDrop Protected
  28. Invasion: PhysicalMemory Protected
  29. Invasion: RawDisk Protected
  30. Invasion: Runner Protected
  31. RootkitInstallation: ChangeDrvPath Protected
  32. RootkitInstallation: DriverSupersede Protected
  33. RootkitInstallation: LoadAndCallImage Protected
  34. RootkitInstallation: MissingDriverLoad Protected
    Score 340/340

(:CLP)

When using “COMODO - Internet Security” preset, the “Invasion: RawDisk” test will fail. Using “COMODO - Proactive Security” preset will be just fine. (Well, I use my custom rules anyway)

#11
COMODO Leaktests v.1.1.0.1

Date 4:33:59 PM - 11/17/2008

OS Windows XP SP3 build 2600

  1. Hijacking: ActiveDesktop Protected
  2. Hijacking: AppinitDlls Protected
  3. Hijacking: ChangeDebuggerPath Protected
  4. Hijacking: StartupPrograms Protected
  5. Hijacking: SupersedeServiceDll Protected
  6. Hijacking: UIHost Protected
  7. Hijacking: Userinit Protected
  8. Hijacking: WinlogonNotify Protected
  9. Impersonation: BITS Protected
  10. Impersonation: Coat Protected
  11. Impersonation: DDE Vulnerable
  12. Impersonation: ExplorerAsParent Vulnerable
  13. Impersonation: OLE automation Protected
  14. InfoSend: DNS Test Protected
  15. InfoSend: ICMP Test Protected
  16. Injection: AdvancedProcessTermination Protected
  17. Injection: APC dll injection Protected
  18. Injection: CreateRemoteThread Protected
  19. Injection: DupHandles Protected
  20. Injection: KnownDlls Protected
  21. Injection: ProcessInject Protected
  22. Injection: Services Protected
  23. Injection: SetThreadContext Protected
  24. Injection: SetWindowsHookEx Protected
  25. Injection: SetWinEventHook Protected
  26. Invasion: DebugControl Protected
  27. Invasion: FileDrop Protected
  28. Invasion: PhysicalMemory Protected
  29. Invasion: RawDisk Protected
  30. Invasion: Runner Protected
  31. RootkitInstallation: ChangeDrvPath Protected
  32. RootkitInstallation: DriverSupersede Protected
  33. RootkitInstallation: LoadAndCallImage Protected
  34. RootkitInstallation: MissingDriverLoad Protected
    Score 320/340

why i have two fails?
11. Impersonation: DDE Vulnerable
12. Impersonation: ExplorerAsParent Vulnerable

my firewall setting is default i’m using CIS latest version :THNK

#12

I did the test using Trend Micro internet Security (what we have on my parents PC). The test did not go very well… (see screen shot for results)

Edit: I noticed I may have set the firewall down a bit (a higher levels it blocked without asking and did not remember anything) so I attach two new images: one with firewall on max, and one with firewall + defense against internet theft on max.

Edit: I used vista x32 on a admin account with Windows firewall deactivated and Windows Defender deactivated and UAC on.

[attachment deleted by admin]

#13

This is not a list, just some thoughts about application and OS settings.

Relatet to applications, all should be tested with default settings and highest security settings.
Because there is a big difference if you test

  • KIS 2009 with default settings (Automatic Mode) or with Interactive Mode.
  • NIS 2009 with default settings or with Advanced Event Monitoring enabled.
  • Outpost with default settings (Optimal Host Protection) or with Advanced Host Protection.
    etc. etc.

Testers should be able to select the best settings for the programs they test and afterwards readers of the results would recognize, that some of their programs are very, very weak with default settings (KIS, NIS, …) and only offer a medium or high protection if they use custom settings.

Related to OS, at least all with the same settings, like for example with

  • XP: Admin, Windows Firewall deactivated.
  • Vista: Admin, Windows Firewall, Windows Defender and UAC deactivated.

With Vista 32bit it makes a real big difference, if you test with Admin account and Windows Firewall & Windows Defender deactivated (110/340) or with LUA and Windows Firewall & Windows Defender activated (250/340).


http://img412.imageshack.us/img412/6841/vistaadminvsluaxd2.th.png

Just as a note. These OS results without any 3rd party security software seem to be pretty weird and I hardly can image, that they are intended by the developers of this test program.

Cheers

#14

I think what Melih wants is the results from the “Real World”
Perhaps not have a guide line to follow, Just list the environment and settings these tests were conducted.

#15

Exactly Kyle… afterall our users are not in a test environment! So how does security work for them out of box in the realworld, that is the question. If everyone is making the same mistake in the realworld and misconfiguring something which is then causing their security to be vulnerable, then these tests will show that, or it will also show providers who can’t provide high security out of box because they haven’t found a way of making it practical for usability.

Melih

#16

CLT v1.1.0.3 = 340/340.

:Beer

edit: xp with adm rights, some restrict policies in secpol.msc. (details in sig.)

#17

using DriveSentry & CIS (firewall only,D+ disabled)

130/340

time to switch back i guess 88)

edit:
update!
pass 100% (340/340) using CIS w D+ (everything on D+/advanced/defense+ settings/monitor settings ticked) (:HUG)

#18

ZoneAlarm Pro with “Program Control” set to High + “SmartDefense Advisor” set to Manual; Firewall - Internet Zone Security - High

COMODO Leaktests v.1.1.0.3
Date 20:58:48 - 18-11-2008

OS Windows Vista SP1 build 6001

  1. RootkitInstallation: MissingDriverLoad Vulnerable
  2. RootkitInstallation: LoadAndCallImage Protected
  3. RootkitInstallation: DriverSupersede Protected
  4. RootkitInstallation: ChangeDrvPath Vulnerable
  5. Invasion: Runner Protected
  6. Invasion: RawDisk Vulnerable
  7. Invasion: PhysicalMemory Protected
  8. Invasion: FileDrop Vulnerable
  9. Invasion: DebugControl Protected
  10. Injection: SetWinEventHook Protected
  11. Injection: SetWindowsHookEx Protected
  12. Injection: SetThreadContext Vulnerable
  13. Injection: Services Vulnerable
  14. Injection: ProcessInject Protected
  15. Injection: KnownDlls Vulnerable
  16. Injection: DupHandles Vulnerable
  17. Injection: CreateRemoteThread Protected
  18. Injection: APC dll injection Vulnerable
  19. Injection: AdvancedProcessTermination Protected
  20. InfoSend: ICMP Test Protected
  21. InfoSend: DNS Test Protected
  22. Impersonation: OLE automation Protected
  23. Impersonation: ExplorerAsParent Protected
  24. Impersonation: DDE Protected
  25. Impersonation: Coat Protected
  26. Impersonation: BITS Vulnerable
  27. Hijacking: WinlogonNotify Protected
  28. Hijacking: Userinit Protected
  29. Hijacking: UIHost Protected
  30. Hijacking: SupersedeServiceDll Vulnerable
  31. Hijacking: StartupPrograms Vulnerable
  32. Hijacking: ChangeDebuggerPath Protected
  33. Hijacking: AppinitDlls Protected
  34. Hijacking: ActiveDesktop Protected
    Score 220/340

(C) COMODO 2008

ZoneAlarm Pro with “Program Control” set to High + “SmartDefense Advisor” set to Auto; Firewall - Internet Zone Security - High

COMODO Leaktests v.1.1.0.3
Date 21:00:05 - 18-11-2008

OS Windows Vista SP1 build 6001

  1. RootkitInstallation: MissingDriverLoad Vulnerable
  2. RootkitInstallation: LoadAndCallImage Protected
  3. RootkitInstallation: DriverSupersede Protected
  4. RootkitInstallation: ChangeDrvPath Vulnerable
  5. Invasion: Runner Protected
  6. Invasion: RawDisk Vulnerable
  7. Invasion: PhysicalMemory Protected
  8. Invasion: FileDrop Vulnerable
  9. Invasion: DebugControl Protected
  10. Injection: SetWinEventHook Protected
  11. Injection: SetWindowsHookEx Protected
  12. Injection: SetThreadContext Vulnerable
  13. Injection: Services Vulnerable
  14. Injection: ProcessInject Protected
  15. Injection: KnownDlls Vulnerable
  16. Injection: DupHandles Vulnerable
  17. Injection: CreateRemoteThread Protected
  18. Injection: APC dll injection Vulnerable
  19. Injection: AdvancedProcessTermination Protected
  20. InfoSend: ICMP Test Protected
  21. InfoSend: DNS Test Protected
  22. Impersonation: OLE automation Protected
  23. Impersonation: ExplorerAsParent Protected
  24. Impersonation: DDE Protected
  25. Impersonation: Coat Protected
  26. Impersonation: BITS Vulnerable
  27. Hijacking: WinlogonNotify Protected
  28. Hijacking: Userinit Protected
  29. Hijacking: UIHost Protected
  30. Hijacking: SupersedeServiceDll Vulnerable
  31. Hijacking: StartupPrograms Vulnerable
  32. Hijacking: ChangeDebuggerPath Protected
  33. Hijacking: AppinitDlls Protected
  34. Hijacking: ActiveDesktop Protected
    Score 220/340

(C) COMODO 2008

No difference of what so ever.

#19

WinXP Home SP3 Running CIS 3.3.55810.432 (FW ONLY) with FW Sec Level Custom Mode and Defence + Level Safe Mode and Configuration Optimum Security
Alongside AVG 8 Free and Comodo BOClean

COMODO Leaktests v.1.1.0.1
Date 04:06:35 - 19/11/2008
OS Windows XP SP3 build 2600

  1. Hijacking: ActiveDesktop Protected
  2. Hijacking: AppinitDlls Protected
    3. Hijacking: ChangeDebuggerPath Vulnerable
    4. Hijacking: StartupPrograms Vulnerable
  3. Hijacking: SupersedeServiceDll Protected
  4. Hijacking: UIHost Protected
  5. Hijacking: Userinit Protected
  6. Hijacking: WinlogonNotify Protected
  7. Impersonation: Coat Protected
  8. Impersonation: DDE Protected
  9. Impersonation: ExplorerAsParent Protected
  10. Impersonation: OLE automation Protected
  11. InfoSend: ICMP Test Protected
  12. Injection: AdvancedProcessTermination Protected
  13. Injection: APC dll injection Protected
  14. Injection: CreateRemoteThread Protected
  15. Injection: DupHandles Protected
  16. Injection: KnownDlls Protected
  17. Injection: ProcessInject Protected
  18. Injection: Services Protected
  19. Injection: SetThreadContext Protected
  20. Injection: SetWindowsHookEx Protected
  21. Injection: SetWinEventHook Protected
  22. Invasion: DebugControl Protected
  23. Invasion: FileDrop Protected
  24. Invasion: PhysicalMemory Protected
  25. Invasion: RawDisk Protected
  26. Invasion: Runner Protected
  27. RootkitInstallation: ChangeDrvPath Protected
  28. RootkitInstallation: DriverSupersede Protected
  29. RootkitInstallation: LoadAndCallImage Protected
  30. RootkitInstallation: MissingDriverLoad Protected

[size=10pt]Score 300/320[/size]

1. I’m wondering why I’m failing on tests 3 and 4 no matter what config I have.
2. Why do I seem to have the Comodo Leatests v.1.1.0.1 when others seem to have v 1.1.0.3???

#20

I attached the latest version for you, Mark. And you need to have CIS in Proactive Security Configuration.

Josh

[attachment deleted by admin]