monkeyman: Thanks for the reply. As far as I've found thus far my suspicions were correct, at least its giving me normals correct enough to work with. I've had to change my navigation method for the Controller I'm working on due to the narrow margins of collision i'm working with, which sweeps until it "can't sweep no' mo'." Constrained by distance, or attempt distance, and I have yet for it to infinite loop on me, thankfully...
Good.. I also found that it helps to go double-precision when dealing with the small adjustments if you want high precision in the sweeps. But to do that "globally" in Bullet costs some performance of course.
My main problem was that the convexSweep does get stuck most of the time - i.e. it says the distance is clear up to 0.5 for example, and when you move there, the next sweep can't go anywhere.
Simply reducing the returned distance is not good enough I think since if your vector is almost parallel to a surface you can still accidentally get numerically too close to something.
So it seems the recoverFromPenetration (or similar) is crucial for stable operation, but that runs into the problem that you then rely on two quite different collision-detection methods (for the sweep and for the discrete collision-tests in recover) which have different numerical margins. This is a horrible can of worms. You need severe worm-poison.