Even if we assume that this statement is tailored to support the worth of AVC testing, it does specify "an anti-virus engine", not an AV product (or even an "AV produt" ), not a firewall, HIPS, hash/whitelist checker, sandbox, virtual machine, etc. So while I understand what you're saying, I also agree with the quote.
You are correct, detection rate might be the most reliable way of testing an AntiVIRUS product yes. But these days antiVIRUS as simply not enough. It might have been enough 10 years ago, but these days you simply need MORE than something that only looks for Viruses. You know that, I know that and everyone else here knows that. But the sad fact is that the average user does NOT know that. To them an antivirus product or an internet security suite is the same. It protects your computer from harm and infections. And AV-comparatives and all the vendors that put the AVC logo on their box exploit this fact that people don't know about computer security.
Let's not forget we are talking about the AVERAGE computer user, not about experienced users like us. If you take a look at the comments about the av-comparatives website in WOT (Web Of Trust) you will see comments like:
Good website for information regarding virus testing as well as anti-virus programs co-operation in a network of threading information and making it available for public viewing. A+.
Good company/group that compares antivirus suites.
One of the most reliable AV test sites. Unbiased and accurate from what I have always seen.
Very informative, a must to visit if shopping for antivirus software.
Just to name a few...and the last comment is what I am talking about. Their tests DO influence what people choose to purchase. Yes I agree that they do a good job in testing how effective an AV product DETECTS a VIRUS, but is that really what people want? Just a product that detects viruses? I thought they were looking for something that keeps their computer SAFE in the end? You honestly believe that people say "I just want something that RECOGNIZES viruses, I am not too concerned about other types of malware and the 3% of malware that is NOT detected by the AV."
It is fascinating to see that the world of preventing and treating diseases is exactly the same in the digital world as in the physical world. Chemo and radiation treatment is THE industry standard to treat cancer and THERE IS NO OTHER way! This has been proven by medical experts, the FDA, the scientists and every 'educated' person you ask. To simply even entertain the thought that there is an 'alternative' cure for cancer it preposterous! If there was, it would have been on TV, in the newspapers, on the internet, everyone would have been notified of this, and certainly the FDA would not keep this from us would they? It is the same in the digital world. People see 'experts' like AVC and others as someone they can trust that they will advise them what to get. Sure people have the freedom to do their homework and look at other sites, but the sad fact is that people do not want to read and learn. They simply want an 'expert' to tell them what to buy. And as you see in these comments about the site, they actually do use that site to decide what AV product they will buy. But is what they buy really going to protect their system as good as other products?
And then there is the question about fairness and honesty about the tested AV products. If it is really unbiased, then why don't they test ALL av products, regardless if they pay thousands of dollars or not? And why don't they give out more details about HOW they test this? And why is there not third part involved in checking on them? Where is the transparency?
I believe the best method of testing AV products is the way Languy99 does it. Simply install an AV product, leave it in STOCK SETTINGS (as most regular users would) and simply go to dangerous sites and run malware (zero day). If the AV product asks you anything, simply answer "YES" without reading (as most average users would) and see what happens. Is your computer infected in the end of the ride or not? Here you can actually SEE what is going on and HOW it was tested. Then you can decide for yourself if you liked what you see or not, rather than look at some statistical numbers on the site. But I guess it takes less time to simply look at the highest scoring number rather than watch all the review video's...