There have been other topics in which the question of why CMF fails some buffer overflow tests is asked. While Tyler Durden has stated that the exact operation of CMF is a secret, he has stated that CMF uses hooks. Here is an interesting article titled 'Bypassing 3rd Party Windows Buffer Overflow Protection' (
http://kd7yhr.org/bushbo/misc/phrack/phrack62/p62-0x05_Bypassing_Win_BufferOverflow_Protection.txt) that discusses how some buffer overflow protection products work. Please note that the article discusses stack backtracing, which Tyler Durden has stated CMF does not use. However, other insights from the article may be valuable. I believe that the reason CMF fails some buffer overflow tests is that CMF detects buffer overflow only when hooked Windows API calls from the shellcode are used. If I am mistaken, Mr. Durden, please correct me.
CMF doesn't backtrace stack frames, it just checks the page corresponding to it's internal page-buffer, so it doesn't vulnerable to fake frames. And again such tests doesn't use any shellcode at all, that's why CMF doesn't "detect" them.
Author