I checked some antivirus reviews, and I’m starting to wonder if there is any sense to make those tests? Why? In February one of the magazines PC-Format reviewed some internet security programs, I can’t remember who “won” but that is no an issue. Important is that one of the best security software- Kaspersky Internet Security results were really poor, it didn’t detect many malicious files… Ok, we know that there is no, and there won’t be antivirus, that can delect 100%( even 98-99% detetcion rate is nonsense).
Last week PC Format has corrected test, and claims that KIS detects 100% of malware! ??? What happened? Kaspersky simply updated virus database(of course with samples used in the test) and had his product retested
So how we know that any test is reliable? we don’t.
I shall bring popcorn and sit back, watching this thread. 8)
There are many opinions… mine is that tests should be considered as some kind of indication, but you have to be very careful. The only thing that is for sure - in my opinion - is default deny, it always works, unless you allow (of course).
I’m sure there will be many wise words in this thread so I’ll step back now and watch the experts and the amateurs (I’m among the amateurs ;)).
True, they don’t have that good detection.
CIMA is hovever pretty nice and catches stuff you throw at it, I will love to see “CIMA heuristic” in cis! =)
The only thing “close” to 100% is default deny… or possible using a sandbox… or not connecting to the Internet at all :-TU :-TU
Sit down, stand up
WARNING: The end is beginnig
Or turning off computer! 
Of course they do make sense, mostly to AV companies but still 
If there are plenty of companies producing AV software, there have to be some tests conducted by unbiased (in principle) organizations/magazines or whatever to judge which product is the best.
Thanks to these tests a potential user can get the picture of which product is good and to which he/she should give a wide berth. If one sees that let’s say G-DATA is always on top in multiple tests, he/she will probably buy it, not another AV.
Just my 2 cents…
You mean Melih’s vs RejZor’s ‘conversation’ (to put it mildly ;D)?
Spot-on :-TU
Yeah, and I’m Chuck Norris ;D. Wanna kick? 
LOL!
I guess I mean any conversation involving Melih. ;D Let’s see if he posts here as well.
Let me brainstorm for a second. :-La
How about this: All AV vendors in the world put all their samples in one huge box, then we test every AV against all these samples. Would that be an accurate test? Well, probably better than any current test out there (remember, I’m brainstorming now, thus also speculating). However, if we gather all samples in the world… there will still be some unknown threats in the wild, and there will be new threats released every day.
Based on the above paragraph I guess my simple conclusion is that no test in the world can provide a 100% accurate and fair picture, but all together, tests can help users determine which AVs are good and which are less good. This conclusion doesn’t come out of rocket science. I am an amateur… but yes, I like technology and methodology.
I guess you’re right Leoni, but consider one thing: many people in the world still believe that Kaspersky is the best AV out there which, I think, is rubbish! It still belongs to ‘the best’ but is no longer a no 1 in my opinion. That’s where the purpose of such tests manifests itself - people should know which AV is at the top.
However, these tests should be unbiased and fair. This is where we come to some problems, I guess ???
This is my point :-TU too ;).
And I want to make one thing clear: I think that AVs as solitary PC defense are a thing of the past now. They are still very useful but should take a back seat, that’s for sure ;).
Thanks
Well lets see what all the AV companies and those Testers have said about current tests:
http://amtso.org/press/20-amtsotestingguidelines.html
“As anti-malware solutions become more complex, many existing tests are unable to evaluate product effectiveness properly, resulting in product reviews that are sometimes incomplete, inaccurate and misleading. The adoption of standards and guidelines is the first stage in an initiative in which members of the industry are collaborating with testers, reviewers and the media in the production of advice that will help customers make better-informed buying decisions.”
And here is the list of all the Companies and testing organisations who are part of this and made the above statement.
http://amtso.org/members/supporters.html
In one hand they make the above statement and on the other hand they continue to launch these tests that they themselves call: “product reviews that are sometimes incomplete, inaccurate and misleading”
One good recent example of this was: None of so called Anti Virus products who got 99% in these tests were able to stop Confiker until they created the signature for it, yet CIS “prevented” it at day zero!
That’s why AMTSO made the above statement: “many existing tests are unable to evaluate product effectiveness properly”.
Melih
I agree with you Melih for the most part :). I assume that you subscribe to this: “many existing tests are unable to evaluate product effectiveness properly”, so how are you going to test CIS so that an average Joe could see if it is good or not and how it compares to the likes of Avira or Kaspersky?
A couple of months ago you wrote that you had a plan for that. Could you say anything more about it?
Thanks
I can’t say yet.
Melih
The results depend on the testing method and of course the samples used. Different tests yield different results. No product can detect everything, and no test is perfect.
The tests themselves are very relevant and provide important information. How accurate they are depends on who is performing the test and how. I think you’re confusing the two.
One good recent example of this was: None of so called Anti Virus products who got 99% in these tests were able to stop Confiker until they created the signature for it, yet CIS "prevented" it at day zero!
This is actually a nonsense we were talking at Wilders. CIS did not prevented it (maybe just the BO part), however as for the D+, users prevented it. D+ just asked what to do. Thats a huge difference.
Also Conficker scenario shows how users really care for their security and which they later bash Microsoft how ■■■■■■ they are. Where in reality, Microsoft provided security patches but these “smarter” ppl are so freakin smart they know it better than those who installed the patches.
If you ask me, updates shouldn’t be optional. They should be enforced strictly in consumer editions of Windows while in corporate editions, they should be optional so admins can have control over whats installed on their workstations. Imagine 99% of these home systems would update months ago and we wouldn’t even be reading about dumb Conficker every freakin day.
Oh, for the 100% detection. No one ever in any test claimed that antivirus which scored 99,99% or 100% in test actually detects 100% of all malware. Thats the point where tests are done right, just uneducated ppl interpret them wrong. This just means it was able to detect 100% of the samples in that test, nothing less, nothing more. So far results from e.g. AV-Comparatives match the real situation.
Those that scored good also fare great in general.
" however as for the D+, users prevented it. D+ just asked what to do. Thats a huge difference."
Default Deny. Even if the user gave no response to this, the action would still be stopped by Defense+.
Oh, yes. And what happens if uneducated user clicks Allow anyway? “Default deny” becomes “Default allow”.
Maybe that will not happen on your system with you in front of CIS, but with casual users you just can’t be so confident to say they will not click ALLOW. It’s just not realistic to look at it, like most of you do.
Users click ignore and allow just so they can get past all the popups as fast as possible and open whatever they wanted to open or run.
I wonder why they have security software installed… 88)
Well, i’m wondering that too. It’s usually because their friends install it for them becuuse they know how careless they are.
I wonder too.
When they run Defence+ & Firewall both in training mode all the time.
Dennis
RejZoR
if your theory was correct about people simply clicking allow, then our forums would be flooded with people saying I have CIS and I have confiker…
As you can see our forums are not flooded with that
Melih
That’s bollox, other AMTSO members also got HIPS and can also theoretically catch very most malware at day zero…