Can't link C/C++ tests to Lua using LuaRocks

Question

TL;DR I'm using LuaRocks for my build system and need to link my C++ tests to Lua, but LuaRocks isn't passing the variables I need.

I have two sets of tests for my rock, one in C++ using doctest and the other in Lua using Busted. Ideally we would only compile the tests when running luarocks test but LuaRocks doesn't pass in LUA_INCDIR, LUA_LIBDIR, etc. or any of the dependency variables (FOO_INCDIR, FOO_LIBDIR) to the command. Unless I can get around this that means I have to build the tests along with the library. Fine. But then I run into a second problem:

Even if I were to build the tests along with the library, on *NIX systems neither LUALIB nor LUA_LIBDIR get passed in. For vanilla installations of Lua I can use -L $(LUA_DIR)/lib -llua. Unfortunately, this breaks with LuaJIT because the library is named libluajit-5.1.a.

This is the relevant portion of my rockspec:

build = {
    type = "make",
    variables = {
        LIB_EXTENSION = "$(LIB_EXTENSION)",
        LUA = "$(LUA)",
        LUAROCKS = "$(SCRIPTS_DIR)/luarocks",
        OBJ_EXTENSION = "$(OBJ_EXTENSION)",
    },
    build_variables = {
        -- ...
        LIBFLAG = "$(LIBFLAG)",
        LUA_DIR = "$(LUA_DIR)",
        LUA_INCDIR="$(LUA_INCDIR)",
        LUA_LIBDIR = "$(LUA_LIBDIR)",
        -- These are both "unmatched variables"
        LUALIB = "$(LUALIB)",
        LUA_LIBDIR_FILE = "$(LUA_LIBDIR_FILE)",
    },
    install_variables = {
        INST_LIBDIR = "$(LIBDIR)",
    },
}

I've tried hacking my way around this many different ways, usually involving fallback values, but none work on all platforms.

LINK_DIR := $(or $(LUA_LIBDIR),$(LUA_DIR)/lib)
LINK_TO_LUA_FLAG := $(if $(LUALIB),-l:$(LUALIB),-llua)

# Link command:
$(LD) -L $(LINK_DIR) $(LINK_TO_LUA_FLAG) [...stuff]

This doesn't work for LuaJIT because the library is named libluajit-5.1.a instead of liblua.a, so the flag would have to be -lluajit-5.1.

I also tried this:

LINK_TO_LUA_FLAG := -l:$(or $(LUALIB),$(LUA_LIBDIR_FILE),liblua.a)

It works on Ubuntu (again, except for LuaJIT) but the linker can't find the file on MacOS for some reason even though I know liblua.a is in one of the search directories I pass in. Maybe the linker installed in Github's MacOS containers doesn't support the -l: option.

Answer 1

I managed to hack my way around it by building the tests along with the library, and making a number of changes to my Makefile. This works because LUA and LUA_DIR are always available. I've tested it and it works for vanilla Lua as well as LuaJIT (both 2.0 and 2.1), on both Ubuntu and MacOS.

First, we determine what version of Lua we're running on, and choose fallback paths based on whether we're running LuaJIT or not:

LUA_VERSION = $(shell $(LUA) -e 'print(_VERSION:sub(5))')
IS_LUAJIT = $(shell $(LUA) -e 'if _G.jit ~= nil then print(1) else print(0) end')

ifeq ($(IS_LUAJIT),1)
    DEFAULT_LUA_LIB_NAME := luajit-$(LUA_VERSION)
    LUAJIT_VERSION := $(shell \
        $(LUA) -e 'print(string.format("%d.%d", jit.version_num / 10000, (jit.version_num / 100) % 100))' \
    )
    FALLBACK_LUA_INCDIR := $(LUA_DIR)/include/luajit-$(LUAJIT_VERSION)
else
    DEFAULT_LUA_LIB_NAME := lua
    FALLBACK_LUA_INCDIR := $(LUA_DIR)/include
endif

FALLBACK_LUA_LIBDIR := $(LUA_DIR)/lib

We then construct arrays containing each of the directory we want to add search paths for. Anything that's undefined or empty expands to an empty string, so e.g. if LuaRocks doesn't pass a value for LUA_LIBDIR it just gets ignored.

# FOO is a dependency we define in our rockspec.
LIBRARY_DIRECTORIES = $(strip $(LUA_LIBDIR) $(FALLBACK_LUA_LIBDIR) $(FOO_LIBDIR))
HEADER_DIRECTORIES = $(strip $(LUA_INCDIR) $(FALLBACK_LUA_INCDIR) $(FOO_INCDIR))

By putting these into an array, we can use the built-in addprefix command to construct the flags for us, avoiding generating flags for variables that are empty or undefined. As a example of what we're trying to avoid, suppose FOO_INCDIR is undefined here:

$(CC) -I $(FOO_INCDIR) -I $(BAR_INCDIR)

The resulting command would be malformed:

gcc -I  -I /usr/include

Now that we've collected all defined directories into arrays, we can build our flags to pass to the compiler:

# If LUALIB isn't defined or is empty, use the library name that we constructed earlier.
LINK_TO_LUA_FLAG := $(if $(LUALIB),-l:$(LUALIB),-l$(DEFAULT_LUA_LIB_NAME))

LIB_SEARCH_FLAGS = $(addprefix -L,$(LIBRARY_DIRECTORIES))
INC_SEARCH_FLAGS = $(addprefix -I,$(HEADER_DIRECTORIES))

# Your compile rule
    $(CC) $(INC_SEARCH_FLAGS) ...whatever...

# Your link rule. Be sure to put all your -l flags AFTER the object files!
target.asdf: $(OBJECT_FILES)
    $(LD) $(LIB_SEARCH_FLAGS) -o $@ $^ -lfoo $(LINK_TO_LUA_FLAG)

The resulting commands that get run looks something like this for LuaJIT:

gcc -I/usr/local/include -I/usr/local/include/luajit-2.0 ...

ld -L /usr/local/lib -L /usr/local/lib ... -lfoo -lluajit-5.1