Tuesday 14 May 2013

C/C++ Build System - Dependencies

So, I'm currently struggling with how I work out what a dependency actually means for my build system. It's a lot harder to work out these things in a C/C++ environment than in a Java one!

Firstly, there's the easy part. A dependency can imply a number of other dependencies. For example, any time you depend on boost-filesystem you also need boost-system. It seems silly to always have to add these two dependencies yourself, when you could have it worked out automatically for you. I know some people dislike transitive dependencies like this, but I think that as long as they can be managed then they are a good thing because they keep your build cleaner.

Secondly, you have the requirement to set up Linker and Compiler flags. These are where it gets a lot more complicated, and you have to be careful what you actually set up.

Compiler Flags come in a few different forms, some easier to handle than others. These can be categorized as follows:

  • Macro Definitions. Essentially any -D flags (Or as appropriate for the compiler) that need to be passed in for the code to compile correctly
  • Include Directories. These are all of the -I flags (Or as appropriate for the compiler) that need to be passed in for the header includes to be found. These will come in two flavours, depending on whether the dependency is in the same project as this one or not. However, the end result is the same - a compiler flag so that the correct header files can be found
  • Everything else. This is the difficult one, but also the one that is less likely to be needed. These flags are going to be different for each compiler, and are going to be compiler dependant. Running pkg-config --cflags for every pkg-config script on my system reveals that the only extra flag is -pthread. That is hopeful at least.
Linker Flags come in a similar set of forms as Compiler Flags, again some easier than others to handle. These are as follows:
  • Directories to find Libraries in. These are all of the -L flags (Or as appropriate for the linker) that need to be passed in for the libraries to be linked against to be found. This has the same issue as the Include Directories, with regards to if a directory is in the same project or not. 
  • Libraries to link with. These are all of the -l flags (Or as appropriate for the linker) that need to be passed in for the libraries to be linked with correctly. These can be difficult, because different systems do different things here. On a Linux system, for example, the shared library needs to be present in the same place at runtime as it was at compile time otherwise the loader can't find it. I don't believe Windows has this problem as long as the DLL is available on the path. There are ways around this, but they are a bit messy. 
  • Everything else. Again this is difficult for the same reason as Compiler Flags, but unfortunately it seems to be more widely used. On my system again, pkg-config --libs lists 5 distinct parameters that come under this category, not all of which I'm sure of what they do without reading documentation. These are -pthread, -rdynamic, -r:, -Wl,--export-dynamic and -Wl,-R. The -Wl ones appear to be broken, as they are parameters for gcc to tell it to pass the rest as a parameter to ld, when you can't always guarantee that gcc is going to be called as a linker instead of calling ld directly.
Finally, there's the complications of working out how to get these settings. The easy case is to have them hard coded in the dependency file, but that gets brittle when you need to work across different systems. Arguably the best option is to support tools like pkg-config, except that not all libraries do support that so that becomes problematic. I'm not sure yet what the best answer is for this...

Saturday 11 May 2013

C/C++ Build System - Simple Configuration

So, I'm writing up what I think would be a decent start for a configuration file for my new build system. It's very inspired by Maven, so the configuration is very similar. I make no excuses for that.

This is an example configuration file that describes a simple project that builds an executable application, and depends on

  • Boost Filesystem 1.49+
  • Boost ASIO 1.49+
  • NCurses 5+

Ideally it will be able to tell that Boost 1.53 - which is the current version - is greater than 1.49 and so will be acceptable, and ideally it will be able to be configured to know that a certain version is too high and isn't suitable.

Because of the difference in how C/C++ projects work to how Java ones work, it will be the case that the dependency definitions are available but the dependencies themselves need to be installed on the system already.

I'm also thinking - because it works well for Maven, and makes sense to me - to have dependency definitions be accessed over the internet from a central repository. That way, you don't need to ensure that you have them on your system in the correct directory for them to work...

C/C++ Build System

I've talked before - here and elsewhere - about the state of C and C++ build systems. And it's not good. Basically, right now, the ecosystem comes down to a whole bunch of disparate tools that are all difficult to use, and that pretty much all involve using a code-driven build script in one way or another (Be it m4 - e.g. Autotools, python - e.g. Waf, or some custom language - e.g. CMake)

So, I'm toying with the idea of writing my own. Probably a stupid idea, and probably one that will never go anywhere, but we'll see. I'm currently playing with the idea of a Maven style build system, where you write configuration file(s) to describe the build and the build system understands that and does the work. This will likely work, similar to Maven, in a plugin-oriented way to make it easy to extend the build system if necessary, but so that you always write your actual build scripts in a declarative configuration file instead of in code...

Watch this space :)