How can one evaluate none perfect security solutions?

Amichai Shulman, Imperva's CTO presented in Source Barcelona today an innovative solution for detecting men-in-the-browser attacks by the web server itself. The interesting aspect of the solution is that it relies on logic that goes unencrypted through the attacker code. As such, there is no way to prevent the attacker from interfering and therefore bypassing detection. On the other hand, it’s reasonable to assume that such bypassing would require special attention from malware creators to the specific detection method and will increase the size and resource consumption of the malware.

So, is it a good or a bad solution? Trying to make this distinction would be hard and may not be valid. A solution has to support a risk management strategy and as such does not have to be perfect. The advantages of the proposed solution might match a specific risk management strategy while the limitations may not be relevant.

While this is true on the conceptual level, I believe two issues hinder this less useful: firstly, to sell a solution vendors need to present it as perfect, limiting the ability of organizations to understand the pros and cons and manage risk effectively. Secondly, there are no good metrics for evaluating the risk so translating the idea of incorporating the pros and cons of a solution into the risk management framework.