There is a problem with Test Driven Development (TDD) and security. Even though I’m a severe proponent of TDD and do my own development (largely) that way, I notice a strong conflict between good architecture and TDD. I’ve also seen mention of this effect in the journals lately, so I’m not alone in this.
What happens is that TDD promotes doing early and minimal implementation, then iterate over it until you get everything to work. Fine, everyone loves that. But early “ready-to-run” code usually implies a simplistic architecture. Not necessarily, but usually, please note.
Now, you start out writing all these tests, ostensibly free from architecture and design assumptions, only specifying the actual requirements. But you aren’t as free from assumptions as you’d like, since just by writing the test in a particular place, you’ve already made an architectural decision. Once the tests are in place and your code runs fine, you’re very free to refactor and improve your code safely, in a kind of localized way, class by class, method by method. But as soon as you do serious changes of architecture, your fine unit tests are usually blown away and have to be refactored or even rewritten. That hurts, and humans try to avoid things that hurt.
After a few incidences like that, you get gun shy and tend to not change your architecture unless you really and truly have to. And there are very few instances where you really have to do that just to make your system work (which is the only criterion your stakeholders care about). So, the architecture of TDD developed systems tend to be monolithic or at least simplistic and kinda smacked together, and guess what… there’s nothing more important than good architecture for secure systems. Forget about buffer overruns and unsafe APIs. It’s the architecture that makes your system fragile or resilient. The rest is just dust and filling. You can make systems run fine and bug free without a solid architecture, but you can never make them really robust.
Personally, I refactor my architecture anyway, over and over, but the only incentive I have is because I’m compulsive obsessive and people tend to not appreciate it (until years later, that is). Every other external incentive there is tells me not to do it. Timeboxing also increases the pressure to leave the architecture unchanged.
So I think we’re in the process of discovering the Achilles heel of TDD: even if the code is great, it much too easily leads to a poor and insecure architecture, and I think we need to take that seriously and try to come up with answers to fix this problem.
And, no, BDUF isn’t the answer.