Memory management.

Post Reply
Black_Moons
Posts: 2
Joined: Mon May 25, 2015 4:20 am

Memory management.

Post by Black_Moons »

Hi. I am having some problems with memory management. Ever since I added bullet physics my game is taking up a lot more ram for its vertex data, and clearing physics data for chunks does not seem to be freeing any memory.

I have so much vertex data that viewing range is actually limited by address space on 32bit builds of the game, and there is a function to reduce viewing range when memory usage exceeds 2.5gigs. This function used to kick in around 800 voxel viewing range and memory usage would not increase from there.

Since I added bullet physics, it kicks in around 400~600 voxel viewing range depending on how much I move around, and viewing range continuously shrinks to nothing as bullet uses more memory despite my game freeing chunks as you move away from them.

Each chunk is 16x16x16 voxels and has its own btRigidBody, btBvhTriangleMeshShape and btTriangleIndexVertexArray. The chunks are meshed into vertices by surface net relaxation. You can play the game at http://www.BrutalNature.com

Some snippets of my code:
This goes on in multiple threads at once, as building the BVH structure was very slow:

Code: Select all

        jobdata->mMeshInterface = new btTriangleIndexVertexArray();

	VoxelVertexType * verticesPtr = &jobdata->mVertexData[0];
	unsigned int * indicesPtr = &jobdata->mIndexData[0];


	btIndexedMesh part;
	part.m_vertexBase = (const unsigned char*)verticesPtr;
	part.m_vertexStride = sizeof(VoxelVertexType);
	part.m_numVertices = jobdata->mVertexData.size();
	part.m_triangleIndexBase = (const unsigned char*)indicesPtr;
	part.m_triangleIndexStride = sizeof(int) * 3;
	part.m_numTriangles = jobdata->mIndexData.size()/3;
	part.m_indexType = PHY_INTEGER;
	jobdata->mMeshInterface->addIndexedMesh(part,PHY_INTEGER);
	//		leafVertexPointersMulti[x] = &mLeafs[x]->mVertexMultiData;
	//		leafIndexPointersMulti[x] = &mLeafs[x]->mIndexMultiData;

	bool useQuantizedAabbCompression = true; // false for speedup generating mesh bounding volumes.. true for less memory usage.
	jobdata->mTrimeshShape = new btBvhTriangleMeshShape(jobdata->mMeshInterface,useQuantizedAabbCompression,true); 

	btVector3 localInertia(0,0,0);
	btTransform trans;
	trans.setIdentity();
	trans.setOrigin(btVector3(0,0,0));
	//mMotionState = new btDefaultMotionState(trans); // Might not need motionstate because its a non moving object? 
	btRigidBody::btRigidBodyConstructionInfo cInfo(0, 0, jobdata->mTrimeshShape, localInertia);

	jobdata->mBody = new btRigidBody(cInfo); 
	//body->setContactProcessingThreshold(m_defaultContactProcessingThreshold);
	jobdata->mBody->setRestitution(0.5f);
	jobdata->mBody->setFriction(0.5f);
	jobdata->mBody->setUserIndex(1); // 1 is for land
	jobdata->mBody->setUserPointer(NULL);
Then in the main thread when a chunk is done updating: (Deletes chunk if it already exists, as chunks can be changed by user input)

Code: Select all

                                  if (dataLeaf->mBody)
						{
							Engine::mPhysics->mDynamicsWorld->removeRigidBody(dataLeaf->mBody);
							delete dataLeaf->mBody;
							delete dataLeaf->mMeshInterface;
							delete dataLeaf->mTrimeshShape;
						}

						dataLeaf->mMeshInterface = (*curVertexJob)->mMeshInterface;
						dataLeaf->mTrimeshShape = (*curVertexJob)->mTrimeshShape;
						dataLeaf->mBody = (*curVertexJob)->mBody;
						if (dataLeaf->mBody)
							Engine::mPhysics->mDynamicsWorld->addRigidBody(dataLeaf->mBody);
And when I detect I need more memory, I call this on chunks that are far away

Code: Select all

 void VoxelOcttreeLeaf::ClearPhysics()
 {
	 if (mBody)
	 {
		 Engine::mPhysics->mDynamicsWorld->removeRigidBody(mBody);
		 delete mBody; mBody = NULL;
		 delete mMeshInterface; mMeshInterface = NULL;
		 delete mTrimeshShape; mTrimeshShape = NULL;
	 }
 }
on the various dataLeaf objects above.

Problem is, this does not really free any memory, and any additional build physics objects take up more memory.

I tried making the physics engine use my own allocator, so I could see whats going on

Code: Select all

void* allocate(size_t size)
{
//	return new char[size];
	EnterCriticalSection(&ThreadSafeAllocatorLock::mCriticalSection);
	void* ret = Engine::mArrayMemoryPoolSafe->Allocate<char*>(size);
	LeaveCriticalSection(&ThreadSafeAllocatorLock::mCriticalSection);
	return ret;
}
void deallocate(void * memblock)
{
//	delete[] memblock;
	EnterCriticalSection(&ThreadSafeAllocatorLock::mCriticalSection);
	Engine::mArrayMemoryPoolSafe->FreeObject((char*)memblock);
	LeaveCriticalSection(&ThreadSafeAllocatorLock::mCriticalSection);
}
Physics::Physics()
{
	btAlignedAllocSetCustom(allocate,deallocate);
... (other physics init stuff here)
But when I went to go check my memory profiling...

I got very nasty results showing more objects deallocated then allocated????

Code: Select all

index, mNumberOfPoolsAllocated, mNumberOfObjectsCurrentlyAllocated, mNumberOfObjectsAvailableSpace, mNumberOfObjectsAllocated, mNumberOfObjectsFreed 
0,0,0,0,0,0
1,1,0,131070,1658,1658
2,1,0,87380,1658,1658
3,1,125,65409,1756,1631
4,1,0,52427,1631,1631
5,0,0,0,0,0
6,1,355,37093,3699,3344
7,0,4294967295,1,0,1
8,0,4294955311,11985,1719,13704
9,0,0,0,0,0
10,0,4294944898,22398,3539,25937
11,1,82,16300,82,0
12,1,226,12880,3485,3259
13,1,4294945590,32627,89,21795
14,1,268,9093,1953,1685
15,1,178,8012,1735,1557
16,1,4294959223,14625,14312,22385
17,1,239,5221,1777,1538
18,8,36703,737,38340,1637
19,1,340,3754,1840,1500
20,1,4294961152,9419,26479,32623
21,13,35242,235,36724,1482
22,1,674,1665,674,0
23,7,12576,1746,14173,1597
24,13,21201,80,36380,15179
25,1,813,551,2367,1554
26,12,13088,940,14455,1367
27,1,844,178,2348,1504
28,29,22924,798,35398,12474
29,3,1403,640,2838,1435
30,3,1244,508,1244,0
31,3,1127,403,2368,1241
32,25,9856,344,21743,11887
33,41,13634,306,14759,1125
34,4,1141,23,2372,1231
35,4,962,54,962,0
36,29,5703,184,17526,11823
37,11,1730,129,1730,0
38,22,3052,138,3052,0
39,29,3577,77,3577,0
40,55,5483,72,14099,8616
41,47,3919,29,3919,0
42,38,2699,37,2699,0
43,29,1760,38,1760,0
44,122,6050,50,12619,6569
45,41,1659,22,1659,0
46,28,949,31,949,0
47,25,725,25,725,0
48,350,8389,11,9488,1099
49,31,613,7,613,0
50,34,566,12,566,0
51,44,604,12,604,0
(Index relates to the size of the objects in the pool, with index51 being objects about 64kbyte iirc, power of 2 down smaller till it gets to a very small size then the pools change by 4 byte in size each)

As you can see, Pool index 7 had one object deallocated but 0 objects allocated! Pool 8 had thousands of objects deallocated and less then that allocated, resulting in god knows what going on.

When I don't use my custom allocator for the physics engine, the memory pool report looks like this:

Code: Select all

index, mNumberOfPoolsAllocated, mNumberOfObjectsCurrentlyAllocated, mNumberOfObjectsAvailableSpace, mNumberOfObjectsAllocated, mNumberOfObjectsFreed 
0,0,0,0,0,0
1,1,0,131070,7182,7182
2,1,0,87380,7182,7182
3,1,342,65192,7473,7131
4,1,0,52427,7131,7131
5,0,0,0,0,0
6,1,1089,36359,15557,14468
7,0,0,0,0,0
8,1,338,25875,7428,7090
9,0,0,0,0,0
10,1,788,17935,15210,14422
11,1,277,16105,277,0
12,1,733,12373,15042,14309
13,1,283,10638,283,0
14,1,895,8466,8182,7287
15,1,553,7637,7547,6994
16,1,1756,4796,9005,7249
17,1,751,4709,7709,6958
18,1,1403,3277,8596,7193
19,1,1027,3067,7924,6897
20,1,2347,928,9508,7161
21,1,1740,989,8593,6853
22,1,2128,211,2128,0
23,1,1545,501,8671,7126
24,3,3444,1467,10199,6755
25,2,2693,35,9734,7041
26,3,3085,422,9740,6655
27,3,2752,314,9698,6946
28,6,4708,200,11266,6558
29,7,4554,213,11376,6822
30,7,4051,37,4051,0
31,8,3618,462,10042,6424
32,14,5483,229,12152,6669
33,14,4544,216,10760,6216
34,12,3464,28,9919,6455
35,12,2935,113,2935,0
36,30,6023,67,18103,12080
37,35,5755,160,5755,0
38,104,14978,102,14978,0
39,102,12831,21,12831,0
40,132,13314,18,13314,0
41,141,11783,61,11783,0
42,113,8098,38,8098,0
43,90,5555,25,5555,0
44,177,8828,22,8828,0
45,134,5463,31,5463,0
46,79,2734,31,2734,0
47,52,1536,24,1536,0
48,57,1361,7,1361,0
49,24,468,12,468,0
50,11,178,9,178,0
51,4,43,13,43,0
As you can see, never are there more objects deallocated(mNumberOfObjectsFreed) then allocated(mNumberOfObjectsAllocated), and never does 'mNumberOfObjectsCurrentlyAllocated' roll around to 4 billion.

For the record, my memory pool class can be found here:
http://pastebin.com/WHTkaQn1

So why is bullet deallocating more objects then it is allocating? Why does it seem memory is leaking so badly? Am I freeing my bullet objects correctly?
Basroil
Posts: 463
Joined: Fri Nov 30, 2012 4:50 am

Re: Memory management.

Post by Basroil »

Sounds like you need to replace the default btAlignedAllocator and btAlignedObjectArray with something more custom. By default the btAlignedObjectArray tries to recycle memory and keep the rest in reserve, which is great if you are constantly building, destroying, and rebuilding objects, but not so much in your case.

Have you tried extending your allocator so it works with btAlignedAllocSetCustomAligned? Perhaps adding a "btFreeReserves" function that compacts btAlignedObjectArray variables to just the currently used memory could help a bit (though won't help with the memory limit... I could have sworn most people gave up on 32bit because there's just too much information in newer games)
Black_Moons
Posts: 2
Joined: Mon May 25, 2015 4:20 am

Re: Memory management.

Post by Black_Moons »

Yea my bigger problem is Bullet seems to be deleting more objects then it is allocating. I really don't get that at all....

Am I deallocating objects incorrectly?

<Edit> Ok I figured out I was allocating as char and deallocating as char*... Yes as a pointer, 4x bigger. that was causing some problems for sure..
Basroil
Posts: 463
Joined: Fri Nov 30, 2012 4:50 am

Re: Memory management.

Post by Basroil »

Black_Moons wrote: <Edit> Ok I figured out I was allocating as char and deallocating as char*... Yes as a pointer, 4x bigger. that was causing some problems for sure..
Good thing you didn't allocate 64bit pointers then! :wink:

If there's one double edged sword that makes C like languages great and horrible it's definitely memory management.
Post Reply