I would like to see CLT expanded further :). As well as leak tests, also include BO tests (as in BO Tester) and maybe add some basic virus testing (if possible). If anything, the virus testing (using eicar) could at least show whether your antivirus is working or not (via alerts).
I think this would make an excellent tool for using on other users computers to test their security software to see if their software is up to the job and if not… recommend CIS to them :). I for one, once the new CIS comes out of beta, am going to reccommend this to the users at my work and what better way to show them how good CIS is (or how bad their software is) by ruuning this kind of tool on their computer(s). Like CLT, it would need to be portable as in just extracting to a folder and running the executable.
The more tests (of all kinds) that could be added, the better.
Some inside info below (with permission from Melih)
As you can see, what Comodo is about to do, is providing tests that will cover not just signatures (like today’s tests - AV-Comparatives and the others), but the whole thing: Prevention!
Sounds interesting 8). So will this “product” be run along side exisiting security software (from other companies) and protect the areas that the existing software doesn’t cover?
I’m no security expert but to have a tool that could “test” every possible way of exploiting an operating system would make a great product. I could see something like this being added into CVA. After the scanned results/tests, CIS could be offered as an alternative or links to other free products that could protect these vulnerable areas.
this will be an online service, where websites will be updated on daily basis with previous day’s results…
you can submit files… and we will check them…
I mean, what will happen if you execute a file that is not detected by your AV? Yep… you get infected!!! so 99% detection is not good enough for 1% of the viruses that cause havoc!!!
Lets do some calculations…
Roughly around 20k-30K of new malware is floating around every month!! (this number is just one view that we have…)
Lets say your AV detects 99% and have 1.7M sigs…you say…wow…look at that…BUT…
lets do some quick calculations to see what that really means shall we (:NRD)…
1% of 1.7M sigs is 17,000. …
Remember we have around 20K to 30K of new malware every month… so this av is missing around
85% of the new malware at 20K new malware month
56% of the new malware at 30K new malware month
This is IF (and this is a BIG IF) the testing platform has ALL the malware… Of course they don’t and they only have fraction of the total malware… Which means the numbers of malware that simply infects the users PCs is much higher than the above number!
Still feeling protected with your AV? Tens of millions of people who got infected last week was feeling safe too (BTW: CIS stopped this sucker on its tracks!!)
Time for the truth to come out and see the “Infection ratio” of new malware against AVs…
If my AV detects 99 % of the malware, means it misses about 300 samples out of 30000 a month, assuming that undetected samples are being added quickly
Or I don’t understand this correctly :-\
you see, the new malware is usually the 1% it misses… (assuming that the testing platform has all the latest malware). So catching 99% of the malware is not that helpful if you, every month, keep missing the new malware. Its a very reactive way to chase malware…you are literally chasing it rather than preventing it.
bottom line is: New malware in majority is not detected! check virustotal stats .
Think it thru… if you are an AV company how do you find out about a new malware? Surely Virus Author’s don’t send a copy to you as soon as they finished coding Malware starts infecting…people report problems…then AV companies say…oh dear… here is a new malware lets create signature…the time it takes for them to create sig etc… millions of unluckly people get infected…
Yes. I understand that Prevention should be the first line of defense. The problem is, that Prevention methods used now very often doesn’t state clearly that they prevented malicious activity, (especially the HIPS modules) where an AV will tell you that the object is bad. (there are FP’s ofcourse)
So as you said lots of times before Prevention–>Detection–>Cure should be our 3 layers of defense
I do not understand how 20K is related to a missing percentage of 85%… also I do not see the relation between 30K and 56 %, can you please explain to me… I am not good in maths, but I want to understand…
Can you expand further on this? So, will the results show that (for example), McAfee offered % protection whereas Norton offered %, etc so users get an idea which security product is the best to use and offers the best protection. What kind of tests can we expect this online service to perform?
The formula is flawed mathematically it’s like comparing Apples and Rocks.
take it to either end of the scale and the flaw jumps out at you.
If only 17,000 new malware was released then you get 100% failure and 0% detection.
go the other way if 1,700,000 new malware was released you get 1% failure and 99% detection,
and what number pray tell could you plug in to get 100% detection, you can’t = broken formula.
If this held water then we should hope for more new malware every month.
Sorry Melih I’m sure it’s just an oversight.
Later
PS. I don’t disagree at all with the point Melih is trying to make. I just can’t crunch the numbers right no matter how hard I bite.
so 17.000 missing signatures stands for an amount of 17K and when there is 20.000 malware (20K) each month and, this means: 17/20*100= 85% of missing signatures for the new malware that month…
If there is 30K malware and 17K is missing, this particular antivirus with 1.7 million sigs misses: 17/30*100= 56.66% of the new malware each month…
I understand the numbers now, the main point goes beyond the numbers: Because it is important to detect new threats, av’s need always to anticipate on a new malware environment each month… and the detection each month can never be 100%, so antivirus solutions are missing malware and people getting infected, eventually people will get disinfected by the antivirus, but than there is some new malware around that the antivirus can not detect and the whole process starts over and over again… a vicious cycle…
But we have to remember that the story Melih told only counts for “new” malware threats, so eventually people will get protection, but the protection is not sufficient for new threats and that is why you need real time prevention like CIS for example.
I agree av is not very good for “new” threats, but still we need av’s, because it tells us what is bad, in that way it is a bit more intelligent than a traditional HIPS… (:WIN)
