Hello, I have taken over an article on techsupportalert called the Best Free Antivirus Software and will be revising it.
What I am trying to figure out right now is how to quantitatively distinguish between good Free Antivirus Software. I have already decided that my main criteria will be in terms of protection from real-world threats, and not other criteria such as straight detection ratios or removal capabilities.
So far I am considering:
AV-Test, with an emphasis on the protection scores. Usability will also be considered. Results Here
AV-Comparatives Real-World Protection Tests, with emphasis on the statistics over time Main page for this here
Dennis Technology Labs Reports on this page
However, I am hoping to draw on numerous credible sources (meaning they somewhat represent real-world protection) in such a way that I can come up with good recommendations backed by data. I know that no test is 100% reliable, and evaluations of security programs are obviously much less reliable than those for other fields, but I would like to try to get around that by querying multiple sources. I will also take into account legitimate reasons for products not participating, and will consider differences between tests with a grain of salt.
Any ideas, links, or comments anyone has would be very much appreciated.
You’ve got a serious project ahead. Wish I could help, but other than the well known labs I have not a clue. Seeing the field of free AV products worth testing is limited you could setup a VM and run the same amount of zero day threats with each vendor under the same conditions over a period of time and draw your own conclusions.
500 zero day samples should be fair for each test. If you need any help gathering them feel free to ask.
My money is on Avira Free Antivirus. Just an opinion.
Thank you. It is true that I could test this myself, but this would only be one test. A bad AV can get very good results sometimes and very bad most of the time. The opposite is true for a good AV. Thus, a cross-section does not give enough information. That is one reason why I’m looking for results from a testing organization. I can look at their results over time and gauge how well the AV does on average, which is really what users should be looking for.
The other reason I would prefer to use results from testing organizations is that if I do a test myself it will not be very convincing to many readers. However, if I reference results from testing organizations this will likely have more weight, and be more convincing. Thus, I am pursuing that route.
AV’s have many aspects as to what they are supposed to do. The one I am focusing on is protection of a user’s computer and information. Cleaning will be largely ignored, although usability and system performance (to a point) must be taken into account. User’s of all different levels of familiarity with complex software will be reading this article. Thus, I must take these aspects into account when providing advice for all of them.
Agreed. That is the difficult part. Many reviews focus largely on the detection ratios, as though detecting a lot of malware which has been purposefully placed on the computer is a good way of making sure it would protect real users in real situations. It’s not. Therefore, I am looking for real-world tests which show product’s protection abilities.
I already have enough information for CIS, and can make good arguments for it already. However, in terms of choosing between the other contenders that becomes difficult as they do not have mechanisms in place for protecting users from nearly all malware, as CIS does by sandboxing unknown files.
Thank you, but I worry about the bias that would introduce in my results. For me the only samples I could access would be those which are publically accessible. Thus, it is possible that bias would creep in because the software with the fastest supporters to submit samples, and how long it takes to process samples, would become factors. However, as I am trying to focus mainly on zero-day results, those results would only confuse the overall findings.
Thus, I will take a look at adding West Coast Labs as well, but I think it would be counterproductive for me to conduct my own personal tests.
I am focusing on is protection of a user's computer and information.
Definitely, the level of protection an AV can consistently offer to the user is VITAL.
An AV's impact on system performance and usability (ease of use/choices of settings) come after that.
AV research labs like VirusBulletin(VB100), West Coast Labs(Checkmark), AV-Comparatives and AV-TEST offer regular testings(including simulated Real-World tests) of different AV products, their reports should provide good data.
As to good free AVs, personally I think they include CIS, BitdefenderFree, AvastFree and AviraFree.
Absolutely, with false positive rate being considered as part of usability.
Thank you for these. I left out Virus Bulletin, and will consider that as well.
These are also the ones I am expecting to find most effective, although there may be some surprise changes after the data is examined. At this point I can’t be sure.
Understood, and I actually agree. In a perfect world this is what I would like as well. However, I am much too busy to double as a reputable testing agency. However, if there are others out there who would be willing to do this for me…
I agree 100%. This is exactly the information I wish I had. However, as I don’t know of any way to find this out for the 5 or 6 products being considered I am forced to follow other approaches.
I assume that you’re getting at default allow or default deny. I absolutely agree that default deny is the best architecture (which is something which will be discussed in my version of the article). However, as I am making recommendations for all users, including those who for whatever reasons will not accept a default-deny architecture, I am forced to also consider methods for how to compare the products which use a default-deny architecture.
This is a very good idea. It would take a lot of work to fully implement (probably more time than I can put into this) but is probably the best approach.
However, this approach seems just as useful for promoting CIS as for better informing my article. Why hasn’t Comodo conducted just such a study? I truly believe that the quantitative results from a well-conducted study of this sort would convince many to try CIS. As a consequence of this I would then also have the results I would be looking for, not that this result was the sole reason I am encouraging you to conduct this study.
But to convince many of those who are currently using other products you would also need to show data to shatter the myth that they are protected as well. Thus, I am suggesting collecting data on your competitors as well.
word of mouth is the best advertising…all these people who see the benefit then go and share it with the others…thats how we achieved millions of installations a month.
power of satisfied users is the most powerful weapon in marketing!
