This is a post I could (and in the past probably would) have made here, but as it was directly concerned with anti-malware product testing, it was a better fit for my Anti-Malware Testing blog, though I suppose I could have reproduced it here. Nevertheless: Mac testing – static versus dynamic touches on a somewhat critical problem in dynamic testing where an operating system includes its own intrinsic detection of known malware. (There are some logical extensions to that problem I’m still thinking about.) The short version would be this. Do you tweak the operating system to disable its internal countermeasures? Circumvent the OS by testing statically in the hope of not tripping the internal detection? Or concentrate time and resources on pre-testing to select samples that don’t trip OS-level countermeasures in order to test on-access performance accurately?
I’m not sure it can be a reasonable position to abandon whole product testing for specific platforms, but it’s clearly going to be a tough call, the more or so if vendors find it necessary to use behavioural detection more consistently in the future. (I guess that will depend on how the threat landscape evolves.)
I think I’m probably going to have to spend a lot more time and thought on this in the near future.
David Harley CITP FBCS CISSP