Build setup for bullet with cmake

[DoZo1971]

Hi,

I'm trying to create a build setup for bullet on Windows 7 and Mac OS X using CMake.
With the accompanied CMakeLists.txt I created a Visual Studio 9 project that loads, compiles and builds debug and release libraries as well as examples. On Windows, these libraries now reside in <developmentpath>/bullet-2.78/build/lib/Debug and -Release.
In my own CMakelists.txt I can find them setting the BULLET_ROOT variable to <developmentdir>/bullet-2.78 and performing find_package(BULLET) but only after i manually copy the libraries to <developmentpath>/bullet-2.78/lib.
This doesn't feel right. Moreover, the names of the debug and release library builds are the same so i would overwrite one of the two copying them to the bullet-2.58/lib directory.

What is "common practice" in the build setup, am I missing something?

Kind regards,

Daniel Dekkers

[Erwin Coumans]

Hi Daniel,

Long time no seen

The common practice for Bullet is to include the source code and build Bullet using your projects build system,
instead of linking against precompiled libraries.

I've never used cmake's find_package myself, so if you figure it out, please send us a patch.

Cheers,
Erwin

By the way:

On Mac OSX and Linux, some people 'install' Bullet system wide using cmake+make install, but I rather not install Bullet system wide.
By the way, for Bullet 3, I plan to add yet another build system next to cmake, called premake. It makes it easier to create Visual Studio project files that you can share with others (relative paths etc).

[DoZo1971]

Hi Erwin,

Good to speak to you as a voice from the past. Dutch or English? Ehm... English.
By the way, do *you* still have anything (even if it's just a screenshot) concerning the infamous zero-gravity biljart "game"? I seriously think it was my first (paying) project. Did we even get paid? I don't remember.

-+- The joys of CMake -+-

CMakes find_package() is a bit hard getting in to.
If you specify only minimal options, ie "find_package(BULLET REQUIRED)", it goes in "Module" mode and calls the findBullet.cmake script that is in the CMake distribution. In your case you can set the BULLET_ROOT variable in advance pointing at the right location so this behavior should normally be enough. But you can also provide more options to find_package() like HINTS or PATHS. Find_package() then goes in "Config" mode and it starts looking for yet another config file (named "<name>Config.cmake" or "<lower-case-name>-config.cmake") that the library should provide. Very few libraries do, including Bullet

http://www.cmake.org/cmake/help/cmake-2 ... nd_package

Anyway, we've been working on this CMake hierarchy for weeks (at this point in time i'm discussing the same issues with the zlib/libpng development group).

Our first approach was to create something like this:

• /development
  • CMakeLists.txt
    /boost_1_47_0
    • CMakeLists.txt
      -sources-
    /libpng-1.5.4
    • CMakeLists.txt
      -sources-
    /bullet-2.78
    • CMakeLists.txt
      -sources-
    /MyLib
    • CMakeLists.txt
      -sources-
    /MyApp0
    • CMakeLists.txt
      -sources-
    /MyApp1
    • CMakeLists.txt
      -sources-
    /build
    • -lib builds of boost, libpng, bullet, MyLib, ...-
      -exe builds of MyApp0, MyApp1, ...-

Where the top level CMakeLists.txt would look like:

• project(Everything)
  ...
  add_subdirectory(boost_1_47_0)
  add_subdirectory(bullet-2.78)
  add_subdirectory(libpng-1.5.4)
  add_subdirectory(MyLib)
  ...
  add_subdirectory(MyApp0)
  add_subdirectory(MyApp1)

With the idea that we could then make CMake configure and generate a platform/compiler independent project file with everything in it. Where changing a configuration setting at this high level would result in a rebuilt of all the libraries (and executables) with the same compiler flags.

Unfortunately, this doesn't seem realistic though, because of peculiarities of the (build processes of the) individual libraries.

So now we are building the individual libraries first and using find_package() to find them. A nice side effect is that all these CMake variables (things like <package>_INCLUDE_DIR and <package>_LIBRARIES) are automatically set in the find<package>.cmake modules and you can use these later on as you traverse through the CMake tree.

You then get something like this for the top level CMakeLists.txt:

• project(Everything)
  ...
  set(BOOST_ROOT /path/to/the/root/of/boost)
  find_package(Boost...)
  set(BULLET_ROOT /path/to/the/root/of/bullet)
  find_package(BULLET...)
  find_package(PNG...) # there is no PNG_ROOT, it looks in /usr so you have to "make install"
  ...
  add_subdirectory(MyLib) # only because you want to see the sources in the IDE, otherwise use find_package().
  add_subdirectory(MyApp0)
  add_subdirectory(MyApp1)

We've got the boost libraries 99% this way (some problems with different architectures on MacOS)

We're working on libpng which is a bit more involved. Libpng depends on zlib, findZLIB.cmake and findPNG.cmake scripts exist but they always seem to look in /usr so you have to "make install" them.

We're working on Bullet. You describe the build process via CMake in great detail in the appendix of your manual (thank you for that, libraries should really take your example for a more pleasant "out-of-the-box"-experience). But at the end there still remains something of a "now what?" moment. Where you have all the libs built in both debug and release but you now want to find_package() them. I figured out that I have to copy them to the $BULLET_ROOT/lib directory but that only works for either the debug or the release versions (having the same names), although there are "containers" (cMake variables) in the findBullet.cmake script for all versions.

So that's where I am now...

Daniel

[Erwin Coumans]

I would recommend to keep things simple, and find_package doesn't sound simple.

Unfortunately, this doesn't seem realistic though, because of peculiarities of the (build processes of the) individual libraries.

Is Bullet one of those libraries causing 'peculiarities'? If so, please tell us what peculiarities Bullet creates for you, perhaps we can fix that.

I figured out that I have to copy them to the $BULLET_ROOT/lib directory but that only works for either the debug or the release versions (having the same names), although there are "containers" (cMake variables) in the findBullet.cmake script for all versions.

If naming/location is an issue, have you considered using a custom command POST_BUILD, copying the Bullet libraries to locations where you want them?
Below is a sample snippet how we copy the glut dlls. You could modify it to do something similar for static libraries (and deal with all the various versions, 32bit/64bit, single/double precision etc):

Code: Select all

				IF (CMAKE_CL_64)
				ADD_CUSTOM_COMMAND(
					TARGET AppBasicDemo
					POST_BUILD
					COMMAND ${CMAKE_COMMAND} ARGS -E copy_if_different ${BULLET_PHYSICS_SOURCE_DIR}/glut64.dll ${CMAKE_CURRENT_BINARY_DIR}
				)
			ELSE(CMAKE_CL_64)
				ADD_CUSTOM_COMMAND(
					TARGET AppBasicDemo
					POST_BUILD
					COMMAND ${CMAKE_COMMAND} ARGS -E copy_if_different ${BULLET_PHYSICS_SOURCE_DIR}/GLUT32.DLL ${CMAKE_CURRENT_BINARY_DIR}
				)

Using cmake can be frustrating, and I prefer premake4. It is easier to configure, and uses Lua for scripting. See my recent blog posting here too. You could check out my experiments here: https://github.com/erwincoumans/experiments
Just click on the Download button to get the latest revision of the source code, open the build folder, click on vs2008.bat and open the solution file.

But I guess you also invested plenty of time in cmake to switch over. So if you can fix find_package for Bullet, please share the solution.
From this issue in the tracker, it seems that find_package works on Mac OSX and Linux, if you 'install' Bullet system wide?
Thanks,
Erwin

[DoZo1971]

Hi Erwin,

Difficult problems.
I wouldn't see the "local machine" aspect as too big of a problem though. I think distributing machine independent (but platform/compiler dependent) project files is too much to ask anyway. There are just too many of them.

Putting energy in two meta build systems would be good i guess. I don't know premake, but i'll take a look. Can you combine CMake and premake in a single distribution without anything clashing (files/directory structures)? Then you could just let the two "compete" for a while and see if premake catches some more momentum. You probably at any time have this "sliding window" of the two most promising (meta) build systems anyway: seperate project files and bjam, bjam and CMake, CMake and premake, etc.

Concerning CMakes find_package(). I just have to do some more work. The findBoost.cmake (called by find_package(Boost)) is pretty convincing, although not 100%. And George van Venrooij (i think you met him) just mailed me his adapted version of findZLIB.cmake, checking that one out.

I know CMakes POST_BUILD custom command, that comes in handy occasionaly. But that still doesn't completely solve the problem of filling the CMake container variables with the right libraries. See attached picture. By copying the release built libraries to $BULLET_ROOT/lib, findBullet.cmake finds the (release) libraries, but not the debug ones (of course). I can't copy them to the same directory because they have identical names as the release ones. Boost solves this problem by versioning the names of their libs, see things like "libboost_regex-vc90-mt-1_47.lib". Which I found very ugly at first, but now, well, at least you can see what it is, and you can place all of them in the same directory. It's a bit of a problem where to stop with this version naming though. For instance, Boost doesn't denote the architecture in the filename, which is ok on Windows but problematic on Mac OS.

Maybe i can solve some things myself looking in findBullet.cmake, i'll keep you posted...

Daniel

[Erwin Coumans]

Hi Daniel,

I have a few more questions/comments on this cmake build system issue:

1) Where did you download the find_package for Bullet?

Unfortunately, this doesn't seem realistic though, because of peculiarities of the (build processes of the) individual libraries.

2) What peculiarities? Does Bullet cause them? Bullet is being used in several larger projects using cmake (including Blender) without problem, so it would be good to find out if Bullet is causing issues.

POST_BUILD custom command... But that still doesn't completely solve the problem ....
I can't copy them to the same directory because they have identical names as the release ones.

3) As far as I know, the copy_if_different command can copy to a different filename, by adding the destination fully. Have you tried that?

COMMAND ${CMAKE_COMMAND} ARGS -E copy_if_different ${BULLET_PHYSICS_SOURCE_DIR}/glut64.dll ${CMAKE_CURRENT_BINARY_DIR}/glut64_release.dll

Alternatively, you can add a post-fix to your library names, so that debug and release (etc) create unique names. Bullet already has an optional feature for that. You could try matching the names/locations to find_package needs?

Last but not least, why can find_package(Bullet) not be fixed, so that it looks in the right sub directory?
If it can find custom filenames for BOOST, it must surely be able to find custom locations (for WIN32/MSVC platform)?
Thanks,
Erwin

[DoZo1971]

Hi Erwin,

1) It's shipped with CMake itself. So, on Windows, you would typically have a directory C:\Program Files\CMake 2.8\share\cmake-2.8\Modules with all these find<package>.cmake scripts in it. Those are the ones called by the commands "find_package(<package>)". Apparently, someone at Kitware wrote a findBullet.cmake for you. Reading it, I now know that it expects you to put all the libraries in $BULLET_ROOT/lib and append "_d" to the debug versions. Then find_package(BULLET) finds them and fills the CMake environment variables accordingly. See attached screenshot.

2) All intermediate steps in the build process that involve some kind of user interaction. For instance for ZLIB you have to first remove a header file after which CMake generates a new one. Of course these steps can be automated (with CMakes PRE- or POST_BUILD commands or Python, anything goes, but only after you figure them out manually first).

3) I don't see any reason why findBullet.cmake couldn't be fixed. Well, actually, it doesn't even have to be broken perse, it's just not communicated what it wants from you. So, ironicly, i'm often spending a lot of time figuring out where find_package() wants me to put the data, instead of the other way around, where the data exists somewhere and find_package() finds it, like the name suggests.

Daniel

[DoZo1971]

Ok, it works.

This is a possible build setup for Bullet and the default Bullet "Hello World" application for Windows and Mac OS X, didn't try Linux but should work.

Pros:
- You have a platform/compiler independent build system (sources and CMake scripts). Only the /build directories are platform/compiler dependent and they are auto-generated.
- You can extend with other libs and add more apps on the same level. If good scripts for find_package(<package>) exist, it should be just a few lines in the top level CMakeLists.txt.
- You get XCode project or Visual Studio solution files as a complete set ("Clear All, Rebuild All") as well as project/solution files for individual apps and libs.

Cons:
- It takes some setup time.
- Have to learn CMake a bit. Maybe more than a bit when you start adding more architectures, targets, platforms.
- (Erwin,... still absolute paths)

Here goes...

1) Create a /development directory (if you don't already have one).

2) Download Bullet and unpack it to /development. We'll call this directory BULLET_ROOT (ie /development/bullet2_78 for the current version).

3) Run CMake UI with source directory BULLET_ROOT and binary directory BULLET_ROOT/build. Press "Configure". CMake UI will ask you if it can create the new /build directory in BULLET_ROOT, yes. Choose your favorite compiler. Check the results and press "Configure" again. After pressing "Generate", you will find a Visual Studio solution or XCode project file named BULLET_PHYSICS in BULLET_ROOT/build. Build it in the IDE (do the "ALL_BUILD" project). You can build 32- or 64-bit depending on your system and debug as well as release versions.

4) You will find the library files (*.lib for Windows, *.a for Mac OS) in BULLET_ROOT/build/lib/... for Windows or BULLET_ROOT/build/src/... For Mac OS. Copy them to BULLET_ROOT/lib but don't forget to append "_d" to the names of the debug versions (CMakes find_package() will expect this later on).

5) Create a top-level CMakeLists.txt directly in /development that looks like this:

Code: Select all

cmake_minimum_required(VERSION 2.8)

project(TopLevel)

if(WIN32)
	set(ROOT_DIR "path/to/development/on/windows")
elseif(APPLE)
	set(ROOT_DIR "path/to/development/on/mac-os")
endif()

set(BULLET_ROOT ${ROOT_DIR}/bullet-2.78) # where to find Bullet
find_package(BULLET REQUIRED) # find Bullet and fill environment variables

add_subdirectory(BulletHelloWorld) # dive in our own code

6) Create a /BulletHelloWorld directory in /development and add a Main.cpp containing some Hello World code (see http://bulletphysics.org/mediawiki-1.5. ... ello_World).

7) Create a CMakeLists.txt in /BulletHelloWorld that looks like this:

Code: Select all

project(BulletHelloWorld)

set(MY_APP_NAME BulletHelloWorld)

# Include path to Bullet include directory, find_package() has set this environment variable...
include_directories(${BULLET_INCLUDE_DIR})

set(MY_SOURCES Main.cpp)

add_executable(${MY_APP_NAME} ${MY_SOURCES})

# Add the libraries to the linker in this project, environment variables set by find_package()...
target_link_libraries(${MY_APP_NAME} ${BULLET_COLLISION_LIBRARY})
target_link_libraries(${MY_APP_NAME} ${BULLET_DYNAMICS_LIBRARY})
target_link_libraries(${MY_APP_NAME} ${BULLET_MATH_LIBRARY})

8) Run CMake UI with source directory /development and binary directory /development/build, configure (twice) and generate. You will end up with an Xcode project or Visual Studio solution file in /development/build named TopLevel. Load/compile/build and run it in the IDE and it will show a bouncing ball (well, just the ASCII version ;-)).

Summing things up, the total hierarchy will look like this:

• /development
  • CMakeLists.txt <- CMake script for top level project, tying everything together
    /bullet2_78 <- BULLET_ROOT
    • CMakeLists.txt <- CMake script distributed with Bullet
      /lib <- where the lib binaries will end up
      /build <- CMakes binary directory, where the libraries will be built
    /BulletHelloWorld
    • CMakeLists.txt <- CMake script for Hello World project
      Main.cpp <- source code for Hello World
    /build <- CMakes binary directory, where the project file and the Hello World executable will end up

Comments welcome, as always...

Daniel