Security tests do they really show us anything?

These are just my thoughts and example with no particular product or test, please if you reply don’t use products or test names, I am just after thoughts on testing in general. Thanks.
IMO security test can be miss leading and I don’t think they can be accurate, no artificial test knows how or where a user is going with his/her computer.
E.G In my example I will use a Security program with 85% Detection, prevention etc.

User one. 100% of his use, luckily lands in the 85% good therefore User one has 100% protection. Nothing but praise for the program.

User two. 100% of his use, unluckily lands in the 15% bad therefore User two has 100% hit with Malware. No praise here.

Same program two very different outcomes, how can any test predict how a computer is used? I don’t think it can. What are your thoughts? Thanks and Kind regards. Please note this is just an opinion and no particular product or testing vendor.

Exactly correct - no matter how good a piece of security software is, humans will find a way to munt it. Security tests are really only valid for the time and space it which they are run. Alter the environment or the location and you can get different results.

You could build a car with 700 safety features and I’ll bet someone will still be able to crash it and damage it, themselves or others with it.

To find out if something is foolproof, add a fool and stir. He’ll find the wiggle space. :wink:

Do they have value? Yes, but only within the context in which they were run. Personal opinion, personal computing habits and personal requirements will flavour any individual users experience with any piece of software.

Ewen :slight_smile:

it really depends on what is the test about, and if there is a test then must have anything to proof
i think if the test dont go for anything specific then what is the point of the test?.
for example, A vs B vs C in a test to see who can keep alive until the last standing
that is ok, i like when the test is “until the end” (in antivirus case, until performance crash)(in firewal case until information is filtered)

but if the test is: A vs B vs C in a test to see how well they perform until the race finish
i dont like this kind of test because dont proof anyting
in antivirus case is like have 1,000,000,000 viruses and see how much do they find
there is no point on that, because even when the pc is infected, if no information is filtered and there is no performance issues then there is no problem

every programs got their weaknesses they test these programs with different viruses each time it is like saying a peice of metal is unbreakable but people try to break it but one decides to melt it lol (why did i say metal sorry if it confuses you or if it doesnt make sense.) ;D

There is no 85%. It’s 100% or fail. Imagine if your OS had 15% of its system calls vulnerable to various attacks. Fail. Attackers don’t care about your 99.99% coverage. They care about the 0.01% that you don’t cover.

No, these tests can’t protect you from user error, but it’s important to get all programmer errors out of the way first.