hey I'm making a space game too (rts)

But one thing with ODE that I noticed is that if you scale a collision object it's a huge performance hit. like at least half the speed or so. It might be the same with bullet too.
I fixed it in that project by scaling my model and then creating the collision model/element to that size. that way there wasn't any scaling done on the collision element and it sped things up quite a bit.
Another thing you could try is making objects further away from the camera(s) have less collision precision.