from AVC http://www.av-comparatives.org/forum/index.php?page=Thread&threadID=1054
Our reply to posts done recently by Comodo:
In our opinion, software evaluations should be provided free of charge to the public.
The costs that arise from the tests and demands from vendors should be covered by the vendors, as it is like an external quality assurance assessment (and they also use it for marketing). That vendors have to pay a fee (which is the same for all vendors, and of course has no influence on results) is publicly known, and also written inside the old methodology document which has been available on our website since at least 2008.
We are legally a Not-For-Profit Organization, which allows us to keep the costs relatively low compared to certain other testing labs, because of the support of the Austrian Government and partnerships with Universities.
When I (Andreas) started doing the public tests in 2003, I did it for free and asked users if they wanted to donate something. Practically no one donated, and based on the high demand for continuing the tests, and the increasing complexity of the tests, I had to start asking all vendors to pay a fee. Nevertheless, my goal of providing independent tests has not changed at all - it allows us to further improve the tests and their relevance for the public.
While what they say is true, why do they not publish their prices? Is it because they don't want the readers their results to see how much they charge?
Comodo, on the other hand, promotes itself as providing AV software for free (although on their website they promote the paid versions very prominently). Comodo gets money for services promoted in the paid version they charge users (which is OK, as Comodo cannot live on fresh air).
Right comodo offers the most free security compared to anyone else in the business, they give it all to you for free, you only pay for the extra services. What other company does that?? None that I can think of off the top of my head.
Melih continuously stated in his forum that we get paid by vendors, and insinuated that the payments have an influence on the results.
This is clearly not the case, as the confidential results we provided to Comodo (and which he published without permission) show.
So how come in your tests other av companies do so well when in my tests most fail miserably? Remember my latest Norton review? Also he would not have published the results until AVC mailed him threatening to expose the results if he did not remove the posts. You are just mad that he beat you to the punch and you have nothing to reveal anymore.
Comodo paid us several times for conducting static detection tests of their software, and although they promised users in their forums that they will publish the results, they never did (they said to us it would be bad for their marketing and therefore they preferred to keep it internal).
True it would be bad for marketing because most users don't understand how to read the numbers. In then end comodo could not publish the results until they paid you more money anyway.
Of course my co-worker got angry about reading such blatant insinuations/misinformation and asked Comodo to clarify the misleading posts in their forum. It has nothing to do with censorship or bullying. If someone says in public wrong things, it might be an opinion/free-speech, but nevertheless not true, and the offended party can try to get it resolved (either by asking for a public apology or by employing a lawyer), in the hope of getting it sorted out in a professional way. However, instead of getting back to us, they preferred to continue with the campaign.
If not bullying, what is up with that e-mail threatening comodo?
We have told Comodo in the past that testing Comodo Internet Security showed it to be very noisy (very many user-dependent pop-ups and lot of false alarms) and that we could not see it working well enough to test it with the automation we had at that time. This was a nice way to say that their software needed improvements. We even suggested how they could word it on their website, without saying that it was due to their scores; they didn't do that at the time, but kept quiet.
what you said is that because of the user dependent pop-ups you are not capable of testing CIS, not that it needed improvement. Having popups does not mean that it could not protect you, actually looking at the trends form other manufactuers they are showing more popups now then ever before because they realize that total automation in today's world is impossible while still offering better user security.
Melih stated that we can only test for detection and not for protection.
This is not true. We test for protection in our Whole-Product-Dynamic Test (which is part of our public main test-series). Unfortunately, Comodo seems not to be interested in joining that test, as the results would be published. At the beginning of the year we wrote in our survey report that although many users would like to see Comodo tested, Comodo did not apply for inclusion in the test.
You actually said you can't test the automatic sandbox, not him. Two things, would you be able to test CIS in the WPDT with the autosandbox or would not be able to? And how much does it cost to test, how many thousands and how much more to publish the results?
We give missed samples AFTER most tests to the publicly tested vendors. Comodo currently pays various other sources for samples and would like to get samples from us for free (they tried to persuade us, using public pressure, some years ago). We do not sell samples.
Yes you do, for the people to get the samples that you tested with they have to pay extra. If that is not selling samples I don't know what is.
Anyway, it is sad to see a vendor trying to publicly bully a testing organization by acting this way: making insinuations in their forum; criticising testing organizations if the results in commissioned tests are not as the vendor would like; publishing confidential/internal email correspondence without permission.
This is more the behaviour we would expect from the CEO of a rogue AV vendor, not from a serious company like Comodo.
Unfortunately, by acting this way Comodo we probably might not test Comodo’s products in any way for the foreseeable future. The reason is simple: if Comodo were to score badly, they would then state that they got poor results because of this "dispute" (and some Comodo fans might even believe that).
The only reason why we will not ask a lawyer to deal with this case is that we prefer to put money into improving tests rather than playing this game with Comodo.
Actually what is sad is a testing organization not having any ethics. Trying to blackmail the other companies to pay more to do anything. I think the only reason you will not go to a lawyer is because you don't have a case because you tired to bully a company in censoring it's users and they exposed you for what you are. The above is my opinion only and does not represent comodo in anyway.