Independant testing of CAV down the road?

I make it a point not to install any antivirus program that hasn’t:

  1. Been independantly tested and
  2. Received a Virus Bulletin 100% award

So it will be a while before I install CAV on any of my systems. I am curious though if Comodo plans to put CAV through such testing down the road when the product is out of beta?

PS…I am not at all attempting to be neagative in the least about CAV. I have not tired it. And in general, I don’t try products in beta because I am not a savvy computer user. I just wanted to know about future plans to test CAV.

Hi,

Currently CAVS is in the beta phase and will most likely not be tested until the final release of CAVS.

The problem is: What is tested?
What do you think is tested?

Melih

For me, tested means reviewed and evaluated by an official organization that exists for that purpose. That organization must be independant and their opinion must not be inlfuenced or biased by the companies whose products they are testing.

For example, VIRUS BULLETIN puts various antivirus software products through a battery of tests, then publishes the results. This is good vor the antivirus industry and good for the consumer. I have provided a link below.

http://www.virusbtn.com/index

Ok, let me try to be more specific. ( i am trying to make a point hence my question so pls bear with me. :slight_smile: )

What do they test exactly?

Do they test that software does not crash?
Do they test how many viruses the product detects?
How do they know how many viruses are out there for them to be able to test effectively to see if the product catches them all or not?

Melih

(PS Don’t get me wrong, I am not knocking them, they do have a use, I am just trying to establish their exact value).

Virus Bulletin spells out clearly on their website what tests they run, how they run them, and how a product receives a VB 100% award. If you are serious about creating a world class antivirus program (and I bnelieve that you are) then you should probably check out their website. I believe that the 100% simply means that your product scored a 100% on their test, not that your product can detect 100% of known viruses. If I am not mistaken, in order to receive a VB 100 award, your product must correctly identify all of the “in the wild” viruses and also not create any false positives. But don’t quote me on that. You can find out the details from their website.

On that topic, as new viruses and malware programs are generated daily, you can never rest assured that your product will catch all the nasties (that is one reason to have a layered defense system, and to use heuristics as well as signatures). But if you can’t at a minimum pass the VB test with a 100% award, then your product still needs work. A VB 100 in my opinion should be a minimum standard. Your comapny may in fact, have higher standards than VB 100. And from my interactions with you, I would guess that you are type of person that wants his product to exceed the best standards out there.

Solo

My questions were not about what VB etc do in general but trying to emphasize the issue about “testing” in general (btw we do have a subscribition to VB already and know almost all certification places for these products.)

The issue I am trying to raise is:

What do you want your AV to do?

1)Detect that there is a virus
2)Stop any malware (virus, spyware, trojan) from getting into your system in the first place?

My point is: AV products in general utilise the “old technique” of looking for viruses on your machine and the testing that is done is based on that.

What is more effective protection? Stop the viruses in the first place or detect them after they are in your system?

Of course its about stopping the viruses/malware on its track!

Now, show me a testing organisation that test that!

There isn’t one!

All the testing is based on measuring detection and not measuring “prevention”.
So can a Product stop every single malware without even detecting a single so called virus from VB?
Of course it can.
Does this mean that this product is bad?
Of course not. It just means that the old methods of testing the effectiveness of Security products in the AV/Malware market is umm… “old”!

Melih

Melih,

I totally agree with you. An ounce of prevention is worth a pound of cure. It is MUCH better to keep the system from getting malware in the first place, But at some level, you need to rely on an end user to make the right decision in order to prevent malware. That’s why we still need to detect and remove because the end user is going to make mistakes.

Ultmately, a system that used both models would be the best. A system that prevents or stops most malware before it ever gets on your machinwe combined with one that will detect and clean what slips past the prevention mode.

I go to sites like Castle Cops, Wilders Security Forum, Spyware Warrior, etc to stay abreast of industry trends and to keep trakc of what security gurus are raving about. The experts (like yourself) know way more than me, so I generally put on my machine the programs that they are raving about. That’s what lead me to put CPF on my system, the guys at Wilders couldn’t give it enough praise. When CAV starts getting those types of reviews, of course I will want to try it out.

I definitely like how CPF evolved and look forward to trying out other Comodo products as they evolve in a similar fashion.

Keep up the good work!

ed

That “independence” of “virusbtn.com” is a thing to be sceptical about, if you look a bit closer at things.

Let me spell it out for you:

Administrative contact: Nicola Mercier Abingdon Science Park

Abingdon Oxon
OX14 3YP
UK
+44.1235559933
+44.1235559935
registrar@sophos.com

Now, isn’t it strange that the REGISTRAR of that domain has a virus security package to SELL you?

I would use better sources than that site. Other than that, I totally agree with you in the following:

This is good vor the antivirus industry
However I'm not certain about the other half of the sentence:
and good for the consumer.

The virus bulleting having economic ties to a manufacturer of antivirus software cannot POSSIBLY be a good thing for the customers. I’m sorry.

//Svein

firstly: Interesting work there Sherlock Svein :slight_smile: We’ll keep that in mind…

There are 3 mechanisms that I would like to employ

1)prevention
2)detection
3)cure

currently AV products don’t really (IMHO) employ prevention. They use signature based detection and some heuristics. These all fall into detection category and not prevention.

I had to make a decision in the AV development. Where do I spend the money in the next 6 months, Improving detection or creating a unique and innovative firewall that will stop even day zero malware… Well, knowing that stopping viruses would be much (i mean MUCH) more effective than trying to catch up with malware writers, I thought, lets turn the table around and instead of “allowing” everything and then once things have infilrated your pc, you go hunt for them, why not just stop them. Thats when i decided to have HIPS functionality in the AV. This has never done before. I think we are the first ones (feel free to correct me)… to do that.
with this, we are going to bring

Prevention into a product that does detection. Perfect!
Of course we will continue to improve the detection but now I will feel more at ease knowing that I have spent the money in the right place cos I know this HIPS based AV we have built will protect our users much more effectively than all the other AVs in the market place.

Also, with HIPS enabled AV, we’ll make sure the user does not have to make many decisions. We have a huge team that we have assembled that is continually increasing our certified list. With this list, we are hoping that majority of the everyday applications will be covered and user’s won’t be bothered by asking them about these already certified applications.

Melih

It only makes sense to lock your door before the intruder gets in instead of trying to fend them off after letting them in. You are doing a wonderful job Melih! :wink:

Paul

I totally agree with you Melih, and I think that you spend the money on the right thing. (:CLP)
BUT, i also think that you will need that “old” detection, to get new users to your product. People read tests in PC-magazines and on the web, and get recommendations on wich software to buy/use. You said it yourself… no one test HIPS/prevention fully, so then you wont get a good review without the old-fashioned detection function… and no new users besides us members… ;D

indeed, we will improve detection and make it one of the best in the markeplace. its just matter of time…

Melih

We all count on that… ;D

Well detecting poly and meta viruses (especially ITW ones!) is a good thing for starters…
Some AV’s are still having very hard time detecting these and there is more and more of such.
AV seriously failing these is not really good to be used for any serious protection.
Thats why ClamAV for example isn’t recommended much even though worm/trojan detection isn’t that bad.