Without testing, I'm not sure what the limits are, in terms of what is and what is not recognised as a 'new' executable.
I only know the rules don't change because I run several applications that are nightly builds, so the executable changes, for each application, everyday. However, the path and the file name remain the same.
For interest, I just performed a small test on a clean system with a fresh install of CIS. I put D+ in paranoid mode and the firewall in custom policy with alerts on very high.
Then using a zip build of firefox 4.b13pre I created a short-cut on the desktop and launched the application with the profile manager active. I created a profile called test and let the application run. It gave alerts for both D+ and the firewall.
I then replaced the files in the firefox 4 folder with those from firefox 3.6.16pre. Nothing else was changed. I launched the application from the same short-cut and selected the test profile. I did not receive any additional alerts.
That's as far as my testing went.
Including a file hash might be a useful addition, providing it can be controlled. Being a beta tester, I really don't want alerts every time something changes. Perhaps a check-box on each application rule to enable of disable the check...