Problems with bullet 2.73, SSE and std::vector

hiker · **Posted:** Tue Jan 27, 2009 12:07 am

Hi,

I tried to update the game I'm working on from 2.68 to 2.73. And while most of the changes were easy to integrate, I ended up with some problems related to aligning btVector3 on 16 bytes boundaries: the std::vector template (at least of Visual Studio) does not support aligned data structure. As a result (since I have btVector3 or derived classes) more or less everywhere, I would have to replace all std::vector with btAlignedObjectArray. That would be annoying, but managable. But with std::vector I get good debugging support from VS - an exception (I mean a breakpoint is triggered) when an out-of-bounds access happens, and the debugger can display all elements of a std::vector object. If I use btAligned... , I can only see the first element, and don't get the additional breakpoints when invalid elements are accessed.

I therefore tried (for now, till I can replace all std::vector) to disable aligning btVector3. I:

re-defined ATTRIBUTE_ALIGNED16(a) --> a in btScalar.h
#undef BT_USE_SSE in btScalar.h

Then it would compile, but crash as soon as a body was added to world. Further debugging showed more sse usage in btDbvt.h - after changing the three DBVT_*_IMPL to DBVT_IMPL_GENERIC, I could finally add bodies to world again. But then I realised that karts can't steer anymore. And at this stage I've decided to get some advise before continuing (though I can't guarantee that this is caused by using SSE somewhere):

Is it possible to compile bullet without needing to align btVector3? Would it be possible to add a single define for that? I can imagine that other people might be using btVector3 in their own application, and prefer (at least for debugging) std::vector.

Thanks a lot!
Joerg

PS: I am aware of the performance benfit of using sse, so I understand the need for aligning

Dirk Gregorius · **Posted:** Tue Jan 27, 2009 9:59 am

The MS implementation of the STL has a problem with aligned data structures. Actually it is only one function that passes an argument by value instead of reference (forgot which one). You might want to consider using STLport which does not suffer from this issue. Alternatively you might want to change the signature of the problematic STL function to get a real quick solution to your problem (though I am not sure whether there are any side effects with this change). Using STLport might be a good solution since it is used in many AAA titles that use STL as opposed to custom implementations of the STL.

sparkprime · **Posted:** Wed Jan 28, 2009 1:02 am

This is what I use to work around the msvc bug:

Code:

// msvc needs a horrible wrapper class because it can't handle aligned parameters
// thus operations like resize() and push_back() cause compile errors
#ifdef WIN32
class SIMDVector4s {

    public:

        struct Stupid { float x, y, z, d; };
        typedef std::vector<Stupid,SSEAllocator<Stupid> > Wrapped;
        typedef Wrapped::size_type size_type;

        void pop_back (void) { wrapped.pop_back(); }

        void push_back (const SIMDVector4 &v)
        { wrapped.push_back(reinterpret_cast<const Stupid&>(v)); }

        void clear (void) { wrapped.clear(); }

        SIMDVector4 &operator[] (size_type index)
        { return reinterpret_cast<SIMDVector4&>(wrapped[index]); }

        const SIMDVector4 &operator[] (size_type index) const
        { return reinterpret_cast<const SIMDVector4&>(wrapped[index]); }

        Wrapped::size_type size (void) const { return wrapped.size(); }


    protected:
        Wrapped wrapped;
};
#else
typedef std::vector<SIMDVector4,SSEAllocator<SIMDVector4> > SIMDVector4s;
#endif

hiker · **Posted:** Tue Feb 03, 2009 2:44 am

Hi Dirk,

thanks for your answer!

Dirk Gregorius wrote:

The MS implementation of the STL has a problem with aligned data structures. Actually it is only one function that passes an argument by value instead of reference (forgot which one).

I think resize().

Quote:

You might want to consider using STLport which does not suffer from this issue. Alternatively you might want to change the signature of the problematic STL function to get a real quick solution to your problem (though I am not sure whether there are any side effects with this change). Using STLport might be a good solution since it is used in many AAA titles that use STL as opposed to custom implementations of the STL.

Well, changing the MS templates is not too good a solution, since we want to make SuperTuxKart as easy to compile for others as possible. STLport sounds like a good option, though I somehow doubt that it has the same debugging support in Visual Studio as MS STL has, esp. being able to see the size and any entry in a std::vector. This is (for debugging) essential for me: we have many std::vector variables, and quite often in debugging we need to see certain values of the X.th entry. I know that Visual Studio can not display all entries of a btAlignedObjectArray (note: we are not only talking of arrays of btVector3 - most of our classes have a btVector3 (or derived from it) variable somewhere, so even the list of karts, list of AI karts, ... - none of them compiles anymore with the latest bullet version).

While I am aware that this is a MS problem, I would really appreciate if bullet could support (via a define) an easy way of disabling the alignment. For now this problem means that we are not able to update to the latest bullet version (till we have a replacement of MS STL). I will most certainly try STLport and see if it works for us.

Thanks a lot for your help!
Joerg
~

hiker · **Posted:** Tue Feb 03, 2009 2:48 am

sparkprime wrote:

This is what I use to work around the msvc bug:

Thanks a lot for your help here. It's a good idea (and would probably solve the 'seeing values in debugger' problem) - but unfortunately we are not only talking of btVector3 std::vectors. We have many classes that use a btVector3 (or derived) element, and any vector of these classes stopped working - e.g. a list of player karts, a list of AI karts, a list of items on the track, ...

Cheers,
Joerg

Erwin Coumans · **Posted:** Tue Feb 03, 2009 3:00 am

hiker wrote:

We have many classes that use a btVector3 (or derived) element, and any vector of these classes stopped working - e.g. a list of player karts, a list of AI karts, a list of items on the track

Feel free to replace the following lines in your local copy of LinearMath/btVector3.h:

Code:

ATTRIBUTE_ALIGNED16(class) btVector3

by

Code:

class btVector3

We just enabled the alignment by default, but if too many developers have issues we just disable it in the next release.
Thanks,
Erwin

hiker · **Posted:** Tue Feb 03, 2009 3:20 am

Erwin Coumans wrote:

Feel free to replace the following lines in your local copy of LinearMath/btVector3.h:

Code:

ATTRIBUTE_ALIGNED16(class) btVector3

by

Code:

class btVector3

I have tried that - well, I defined

Code:

ATTRIBUTE_ALIGNED16(a) a

in btScalar.h, but unfortunately that's not the only problem. I also had to

Code:

#undef BT_USE_SSE

in btScalar.h, ad then I had to

Code:

//hiker #define DBVT_SELECT_IMPL                DBVT_IMPL_SSE
//hiker #define DBVT_MERGE_IMPL                 DBVT_IMPL_SSE
//hiker #define DBVT_INT0_IMPL                  DBVT_IMPL_SSE
#define DBVT_SELECT_IMPL                DBVT_IMPL_GENERIC
#define DBVT_MERGE_IMPL                 DBVT_IMPL_GENERIC
#define DBVT_INT0_IMPL                  DBVT_IMPL_GENERIC

in btDbvt.h. Then it finally compiled, but had very strange behaviour (e.g. karts would not steer anymore). I was running out of time at this stage, so I stopped debugging any further, but couldn't help the feeling that it might be alignment/SSE optimisation related as well (though it might be any other change from 2.68 to 2.73 as well, which I might have missed).

Quote:

We just enabled the alignment by default, but if too many developers have issues we just disable it in the next release.

I understand that, it's just that it might not be that easy to disable it - if it would be only the single change I wouldn't have bothered starting this thread

Thanks a lot for your help!
Joerg

Erwin Coumans · **Posted:** Tue Feb 03, 2009 3:44 am

Please just change btVector3.h as I mentioned before.

Then make sure to do a clean rebuild all. It seems that the vehicle/forklift demo works just fine with those changes.
Hope this helps,
Erwin

(edit -> no other change should be necessary, leave btScalar.h unmodified)

hiker · **Posted:** Tue Feb 03, 2009 4:12 am

Hi,

I tried that before - but then I have the same problem because btVector3 uses __m128, which is a sse (and I assume therefore aligned) data type.

I tried your 'in between edits' suggestion (remove alignment on btVector3, comment out #define BT_USE_SSE). I then had the same problem with objects using btQuaternion. So I then removed the alignment for btQuadWord, and I could compile everything, but still have the steering problems (though that might be a problem of STK, i.e. something might have changed in bullet that I have to adjust, so I am happy to debug this myself for now).

I try again to only change btVector3 (and leave BT_USE_SSE in), perhaps I had a typo somewhere.

Cheers,
Joerg

DannyChapman · **Posted:** Tue Feb 03, 2009 2:15 pm

hiker wrote:

STLport sounds like a good option, though I somehow doubt that it has the same debugging support in Visual Studio as MS STL has, esp. being able to see the size and any entry in a std::vector.

You can customise the way types are displayed in Visual Studio by editing:

Common7\Packages\Debugger\autoexp.dat

It's a bit cryptic, but I'd be surprised if someone hasn't already worked it out for STLport.

Erwin Coumans · **Posted:** Tue Feb 03, 2009 8:21 pm

Indeed, can you please try to use stl port first, or btAlignedObjectArray, without any Bullet modifications?

Below is the snippet that adds MSVC debugging support for btAlignedObjectArray and btVector3, similar to what they did with stl port visualization. Just add the text in C:\Program Files\Microsoft Visual Studio 8\Common7\Packages\Debugger\autoexp.dat (for example at the very end).
This snippet

Code:

   btAlignedObjectArray<int> vec;
   vec.push_back(2);
   vec.push_back(6);
   vec.push_back(4);

   btAlignedObjectArray<btVector3> vecs;
   btVector3 tmp(6,5,4);
   vecs.push_back(tmp);
   vecs.push_back(btVector3(4,5,6));

looks in the MSVC debugger like this:

Code:

[Visualizer]
;------------------------------------------------------------------------------
;  btVector3
;------------------------------------------------------------------------------
btVector3{
   children
   (
       #array
       (
         expr :   ($c.m_floats)[$i],  
         size :   4
      )
   )
   
    preview
    ( 
        #( 
         "[", 
            #array
            (
            expr :   ($c.m_floats)[$i],  
            size :   4
         ), 
         "]"
      )
   )
}
   
;------------------------------------------------------------------------------
;  btAlignedObjectArray
;------------------------------------------------------------------------------
btAlignedObjectArray<*>{
   children
   (
       #array
       (
         expr :   ($c.m_data)[$i],  
         size :   $c.m_size
      )
   )
   
    preview
    ( 
        #( 
         "[", 
            $c.m_size ,
            "](", 
            
            #array
            (
            expr :    ($c.m_data)[$i],  
            size :    $c.m_size
         ), 
         ")"
      )
   )
}

Thanks,
Erwin

hiker · **Posted:** Wed Feb 04, 2009 2:26 am

Hi,

whow - that's really great news. I tried the 'visualisation' for btAlignedObjectArray and btVector3, and it worked (and I wish I had had it earlier). Now we only have to decide which template library to use, and replace 200+ std::vectors

Thanks a lot for your help!
Joerg

PS: For what's it worth: I was indeed not able to get it to compile with only changing btVector3.h.

sparkprime · **Posted:** Wed Feb 04, 2009 6:02 am

std::vector is sufficiently over-engineered to already support alignment via the allocators framework. The only problem is msvc is buggy but you can work around that by casting.

Physics Simulation Forum

Problems with bullet 2.73, SSE and std::vector

Who is online