I was unaware that this was how it was done until we were acquired by COMODO and I got tossed into that bucket there and got to work with third world veterans of the antivirus business who had worked for various other major names before coming to COMODO. First engineering question I was asked was “how do we detect a packer in a file?” Ummm … entropy? Apparently the AV industry had just started working on that. I won’t disclose whose competitor’s AV engine they stole, but it’s out there for googling.
Not a chance of malware submissions getting any detailed analysis unless someone knew what the MD5 for the file was (yeah, copy and paste THAT) or it was properly “cursed” by enough other AV’s who gave it one of dozens of different names that rang a bell. I kid you not! And you had to search manually through the definitions database knowing what you were looking for to have a look assuming they bothered to even keep a copy of the sample in the first place! So analysis was rarely done unless it was reported back as a false positive problem and the customer remembered what it was detected as.
And there’s yet another fail mode. Someone reports a file as an FP. How is it handled? Why it’s REMOVED from the signatures and tossed back into the “undetecteds” in the database with hope that someone who is bored might have a look at it. What are the chances? And so the criminals learned quickly that the best way of keeping their malware undetected if they couldn’t just go to Jotti or VirusTotal and keep modifying it until nobody detected it or even better, was to report it as an FP to everyone. Problem solved for criminals!
With over 70,000 samples a day coming in the door, analysts just can’t be bothered. And with the public completely used to the idea of false positives being commonplace, if only a handful of AV’s detected their submission to Jotti, then it must be their AV doing an FP. And there’s your antivirus. At work.
I won’t even bother to go into what gets hired over there in those countries. At COMODO, there were plenty of people in the AV labs who were dodgy and a few very good people who just walked away. I was told many tales of analysts who “lost things” as far as detected malware went, and it’s entirely possible that with these overseas analysts that some are in cahoots with the criminals.
I guess it’s simply a matter of whether or not you “trust” these offshore “labs” that everyone’s using to control their costs. And my paranoia about this was not served well at COMODO when the BOClean code that I gave them was sent off to Chinese coders who would not EVER let me audit my own code after they’d changed it.
And in the end, not one single innovation in BOClean made it into COMODO’s product other than the most rudimentary part of reading memory, and perhaps our ability to unhook some things from the kernel. BOClean wasn’t what it was because it could parse system memory properly instead of a file, it was what it was because of the method of creating the signatures for it. Getting into the mind of the perpetrators and seeking out what was unique about the author and using THAT for the signature.
We knew the bad guys would write more code and by focusing on HOW they coded, we were able to successfully detect future zero days. COMODO just went and treated it like any other AV file engine and never grasped the method behind our code. BOClean’s prior reputation was utterly destroyed in the process because it never was built on the traditional AV methodology in the first place and our former customers spotted the difference immediately. In addition to stopping the malware, BOClean also cleaned up ALL of the mess it created in the registry, file system and removed all entrails. FULL cleanup of a mess is something that just doesn’t happen anymore.
But if nothing else, learning how to do AV “properly” at COMODO was a rude awakening indeed. About the only advice that COMODO took from me when I came on board was the concept of whitelisting as about the only viable way of knocking down malware since blacklisting has always been a complete failure. But they didn’t get that right either, because graylisting (we don’t know what this is and we won’t let you run it until we do) was never part of the process and whitelisting properly was too big a challenge. The other AV’s are now following that same course the same way COMODO began to do.
“Reputation-based” protection doesn’t work either in the real world since it depends on people voting intelligently and the ballot box not being trolled by ne’er-do-wells. When 4chan sent Justin Bieber to North Korea and talent got whacked on “American Idol”, there’s a perfect example of how “public voting” works in real life situations. I wouldn’t trust the denizens of Tumblr or the Lulzboat either to decide what’s “safe”. And at COMODO, I learned quickly how “trusted vendors” lists aren’t.
http://www.infosecisland.com/blogview/15106-The-Demise-of-the-Antivirus-Industry.html
