Valkyrie Test

miloszcz · August 9, 2011, 8:07pm

Hi.
I’ve just done my Valkyrie test.

I was testing 2 things:

False positives (on 150 legit exe files)
Detection test (on 200 malware samples)

Safe application set:

I collected 150 safe exe files. In general about 80-90% of these files were system’s files.
Other was from various applications like:

ashampoo buring studio
wtw (IM software)
Comodo Time Machine
Expressivo
FreeYouTubeToMP3Converter
Comodo programs manager
iSpy
SpyShelter
few drivers.

Unsafe application set.

Collected 200 unsafe files. They were fresh, some PUPs were in this set.
I was very strict in rating and Valkyrie didn’t pass some of those, because only its heuristics detectors didn’t catch file (AV expert or CAV, CIMA said that’s malware - I didn’t take this into account,I was testing only heuristic).

Results

SAFE APPLICATIONS

Total safe files: 150
Rated correctly by Valkyrie: 148
Score: 98.66%

Comment: Valkyrie is very reliable in detecing safe applications, false positives are very rare.

MALWARE APPLICATIONS:

Total malware files: 200
Rated as malware by Valkyrie: 170
Score: 85%

Comment: It’s very good score, detection 85% of unknown malware. Test was strict.

Safe files rated as dangerous by Valkyrie:

http://valkyrie.comodo.com/Result.aspx?sha1=629a92efe4240f5ee4a3b540f5cec50cb8dd5bd3&&&query=0&filename=blastcln.exe
http://valkyrie.comodo.com/Result.aspx?sha1=3ade035011c02d02f470bd1ae125978584ec678b&&&query=0&filename=mpnotify.exe

Malware files rated by heuristic improperly:

NOTE: I made comment with feedback to all files I’ve submitted. In total 350 comments for today :P0l

Thanks.

GakunGak · August 9, 2011, 8:13pm

Let me be the first to say that this what you posted is simply AMAZING man!!! :-TU ;D

DiSP · August 9, 2011, 8:15pm

Great job morphiusz and thanks for sharing this information with us.

_0M0D0 · August 9, 2011, 8:29pm

Great work Morphiusz!
Amazing job you did today.

You should definitely enter the COMODO Hall of Fame!

Thanks for helping Valkyrie grow faster and better.

GakunGak · August 9, 2011, 8:31pm

+1 :-TU

wasgij6 · August 9, 2011, 8:38pm

valkyrie is definitely getting to be great. with some more work it will be a beast and nothing will get passed it
great job comodo

siketa · August 9, 2011, 8:42pm

Great job, Morph! :-TU

languy99 · August 10, 2011, 12:40am

good job, can’t wait for valk to be integrated into CIS.

siketa · August 13, 2011, 12:00pm

I quoted your test at Ultimativna antivirus tema - P&O (Sigurnosni softver) @ Bug.hr Forum

Hope it’s ok with you…

miloszcz · August 13, 2011, 12:11pm

Sure :-TU

sdfsadfdfs · August 13, 2011, 1:05pm

by the way,anybody knows when would Valkyrie be added into cis???

GakunGak · August 13, 2011, 1:09pm

You defend Comodo nicely :-TU :-TU ;D ;D

Ajde, da te malo naučim neke stvari.... http://www.bug.hr/d.tiny_mce/plugins/emotions/img/tr1.gif

;D ;D ;D

siketa · August 13, 2011, 1:12pm

Thanks, Gakun…just doing my best to inform people and give them real facts about Comodo and not some blatant lies like Comodo haters do…

GakunGak · August 13, 2011, 1:14pm

Yeah, those people just repeat like parrots what OTHERS told them without trying or verifying.
I applaud you for your effort, a really nice job :-TU

siketa · August 13, 2011, 1:26pm

Yeah…and when we tell them to make their own test all you can hear are excuses about lack of time, lack of certificates…blah, blah, blah…

GakunGak · August 13, 2011, 4:41pm

They like shiny graphics and charts
For example, compare this
http://www.eset.com/us/home/compare-eset-to-competition
and this
http://performancechallenge.norton.com/
;D

Citizen_K · August 13, 2011, 6:02pm

Seeing is believing… :-X

khanyash · August 13, 2011, 7:39pm

Amazing Test Morphiusz.

Regards
Naren