Playing with the tools build system

The residualvm-tools repository was born on April, 7th this year. From the very first moment I felt its build system was broken, mainly for these three reasons:

1. to build a specific tool you must run “make tools/foo”. The repo is called residualvm-tools, there should be no need to specify “tools” in the make call too;
2. there isn’t a single line of Makefile code shared between the different tool targets;
3. each submodule of tools/, like tools/lua/module.mk, needs to be added in Makefile.common. I’d like a way to set it up into tools/module.mk, with a parent-child relationship.

So some days ago I decided to try to improve it a bit, even though I never wrote a single line of a Makefile.

My first try was to work on rules.mk: I modified a bit the code that created a target from the value of TOOL_EXECUTABLE and of some other variables. After some trial and error I came up with a piece of code that allowed me to put in tools/module.mk, for each tool, something like this:

TOOL_EXECUTABLE := unlab
MODULE_OBJS := unlab.o
include $(srcdir)/rules.mk

TOOL_EXECUTABLE := vima
MODULE_OBJS := vima.o
include $(srcdir)/rules.mk

....

Quite ugly, but still better than before.

Then I started working on 3. Long story short, I couldn’t come up with an acceptable solution.

Then an heretic thought passed through my mind: CMake!
I really like the add_executable and add_subdirectory functions of CMake, but I know that porting to CMake would require a big effort, since it doesn’t work out of the box on all the platforms configure currently supports.
So I started thinking: configure works perfectly, I just need a way to confortably generate a bit of Makefile code. configure is an sh script, why not just use sh?

So I wrote a script, makegen, with the functions add_executable and add_subdirectory.

add_subdirectory() {
    MODULE="$MODULE/$1"
    . $srcdir/$MODULE/module.mg
    MODULE="${MODULE%/$1}"
}

add_executable() {
    echo -n "$1_SOURCES := " >> $builddir/Makefile.modules
    for f in $2; do
        echo -n "$MODULE/$f " >> $builddir/Makefile.modules
    done
    echo >> $builddir/Makefile.modules

    echo -e \
"build: $1
$1_DIRS := \$(sort \$(dir \$($1_SOURCES)))
$1_dirs:
    \$(QUIET)\$(MKDIR) \$($1_DIRS)
$1: $1_dirs \$($1_SOURCES)
    \$(QUIET)\$(MKDIR) \$(DEPDIR)
    \$(QUIET_CXX)\$(CXX) \$(LDFLAGS) $3 \$(DEFINES) -I. -I\$(srcdir) \
    \$(filter %.cpp %.o, $+) -o \$@\$(EXEEXT)
EXECUTABLES += $1\$(EXEEXT)
clean: clean/$1
clean/$1:
    \$(QUIET)\$(RM) \$($1_SOURCES) $1
    \$(RMDIR) \$($1_DIRS) \n" >> $builddir/Makefile.modules
}

This script is called from configure and it simply generates a file Makefile.modules with the targets for the tools.

Now, adding a new tool is simply a matter of adding in tools/module.mg

add_executable toolname "toolname.o foo.o bar.o"

You’ll then be able to call make toolname.

This is not finished and still experimental, and I can’t know if it will ever be actually used, but I would like to hear some thoughts.