Yeah, you need DX10 and Physics turn up. Then you can just turn on Strength mode and bash in most buildings that're made out of wood. The concrete foundations can't be punched down, obviously, but it's still satisfying to see what you CAN destroy. =]
I am currently running DX9 on high and can still knock down buildings.
So not necessarily always in DX10.