file generated by add_custom_command cannot be found within add_subdirectory

Question

I'm in the process of generating an api using flatbuffers within cmake. Now when on top level of the cmake directory tree the api is generated, the include files that should be generated are not found when they are needed in the branches.

./CMakeLists.txt

set(INTERFACE_FBS ${CMAKE_CURRENT_SOURCE_DIR}/fbs/testapi.fbs)

flatbuffers_generate_headers(TARGET incl
                             SCHEMAS ${INTERFACE_FBS}
                             FLAGS --scoped-enums --cpp-std c++17)
target_link_libraries(incl INTERFACE flatbuffers::flatbuffers)
set_directory_properties(PROPERTIES ADDITIONAL_CLEAN_FILES incl)


#add_executable(testapp app/src/test.cpp)
#target_link_libraries(testapp incl)

add_subdirectory(app)

./app/CMakeLists.txt

add_executable(app src/test.cpp)
target_link_libraries(app incl)

Running the system like this results in the following error:

CMake Error at app/CMakeLists.txt:2 (add_executable):
  Cannot find source file:

    /home/xxx/build.test/incl/testapi_generated.h

  Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
  .hpp .hxx .in .txx

The testapi_generated.h can't be found as it's not generated. But it should be generated as the function flatbuffers_generate_headers sets an add_custom_command for the wanted .h file.

When I move the building of the executable to the main cmake file, so replacing the bottom of: ./CMakeLists.txt

add_executable(testapp app/src/test.cpp)
target_link_libraries(testapp incl)

#add_subdirectory(app)

the '.h' file is generated and the simple application builds without problems. Also enabling the subdirectory at that point will result in a working executable. Performing a make clean and the building will fail again.

What am I doing wrong? Is it not possible to first use files created by an add_custom_command in a subdirectory?

I've put the whole project into a repository: https://github.com/CedricSchmeits/file-generation-issue

Answer 1

As Milan Š. already stated the problem is caused by scoping. Within flatbuffers_generate_headers the files are generated by add_custom_command which is the correct way of generating files. Within the desciption of add_custom_command this is already addressed with: same directory (CMakeLists.txt file)

This defines a command to generate specified OUTPUT file(s). A target created in the same directory (CMakeLists.txt file) that specifies any output of the custom command as a source file is given a rule to generate the file using the command at build time.

Further on the solution is stated:

Instead, use the add_custom_target() command to drive the command and make the other targets depend on that one.

This means that implementing another target which is depended on all the custom commands created by flatbuffers_generate_headers would solve the problem. Then within another directory the target there should also be made depended not only of the interface library but also of the additional target. using add_dependencies

This is solved in the pull request for flatbuffers: Added GENERATE_ to flatbuffers_generate_headers which is added to release v23.3.3.

The flatbuffers_generate_headers now not only generates the INTERFACE library ${TARGET} but also a new target named: GENERATE_${TARGET}. Which results that only within the sub directory a change needs to be done.

./app/CMakeLists.txt

add_executable(app src/test.cpp)
target_link_libraries(app incl)
add_dependencies(app GENERATE_incl)

Also make sure that you are using at least version 3.20 from cmake as within that version some issues with GENERATE files flags are solved.

./CMakeLists.txt

cmake_minimum_required(VERSION 3.20)

Answer 2

TL;DR: It's a scope issue, and the "solution" outlined in this answer is not really a solution but a hack which I don't recommend.

---

This is a scope issue. The way flatbuffers_generate_headers is written prevents the subdirectory to invoke the correct (sub)commands in order. There is nothing you can do about this, other than "hardcoding the subdirectory in" - which I don't recommend.

What I mean by that is essentially the equivalent of copy pasting it into the root CMakeLists.txt, but in a more hidden way, i.e.

By directly including the subdirectories CMakeLists.txt file:

include(app/CMakeLists.txt)

However down the line this may lead to other issues because the main benefit of add_subdirectory is the scope it creates for the CMakeLists.txt.

Hence I will repeat myself: I do not recommend the above and instead recommend moving either code respectively into the appropriate CMakeLists.txt