How to debug a libtool project with vscode?

Question

I have a C++ project that uses libtools. When libtool builds a program, it places the binary in a subdirectory (.libs), and places a wrapper script with the same name in the build directory, like this:

project/
+- libtool
+- build/
   +- a.out           <<< this is a script, not a binary
   +- .libs/
      +- a.out
      +- lt-a.out

In order to debug a libtool program from the command line, you need to invoke:
../libtool --mode=execute gdb a.out

However, it is impossible to debug build/a.out directly, because it is not a binary. I found that I can debug the program inside vscode, if I point the launch configuration to the .libs/a.out file, like so:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [        {
            "name": "(gdb) Launch",
            "type": "cppdbg",
            "request": "launch",
            "program": "${workspaceFolder}/build/.libs/a.out",
            "args": [],
            "stopAtEntry": false,
            "cwd": "${workspaceFolder}/build",
            "environment": [],
            "externalConsole": false,
            "MIMode": "gdb",
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                }
            ]
        }
    ]
}

Is there a way to invoke the binary through the libtool wrapper?

Is there any advantage to doing so?

Answer 1

To debug a libtool program with vscode, you need a wrapper script that shuffles the command line used to launch the debugger, and make a few changes to the launch.json configuration.

Here is a wrapper script I use (vscode-lt-launch.sh):

#!/bin/bash
# vscode-lt-launch.sh
# libtool wrapper for vscode debugging

# Copyright (c) 2023, Radu Hociung
# This file is in the public domain, free to use for any purpose.
#
# Will be called with the following args, which are re-ordered to create the
#  correct libtool execution command line:
# --interpreter=mi {miDebuggerArgs} --tty=/dev/pts/17
# 
# The VScode launch.json configuration should contain the following fields:
#
# "miDebuggerPath" should be set as "./vscode-lt-launch.sh" (this file)
#
# "miDebuggerArgs" should be set as 
#             "gdb ${workspaceFolder}/libtool {program-name-libtool-wrapper}"
#             in launch.json
#
# "program" shoud be set to "src/.libs/{libtool-built-program-name}"
#
# "additionalSOLibSearchPath" shoud be set to "lib/.libs/" or wherever your
#             program's own shared libs are, if your software also
#             builds shared libs

# Example VScode launch.json configuration (the program name is "myprogram"):
#        {
#            "name": "(gdb) Launch",
#            "type": "cppdbg",
#            "request": "launch",
#            "program": "src/.libs/myprogram",
#                       ^^^^^^^^^^^^^^^^^^^^^
#            "additionalSOLibSearchPath": "lib/.libs/",
#                                         ^^^^^^^^^^^^
#            "args": ["-4", "-myprogram-args"],
#            "stopAtEntry": true,
#            "cwd": "${workspaceFolder}",
#            "externalConsole": false,
#            "miDebuggerPath": "./vscode-lt-launch.sh",
#                              ^^^^^^^^^^^^^^^^^^^^^^^
#            "miDebuggerArgs": "gdb ${workspaceFolder}/libtool src/myprogram",
#                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#            "MIMode": "gdb",
#            "setupCommands": [
#                {
#                    "description": "Enable pretty-printing for gdb",
#                    "text": "-enable-pretty-printing",
#                    "ignoreFailures": true
#                },
#                {
#                    "description": "Set Disassembly Flavor to Intel",
#                    "text": "-gdb-set disassembly-flavor intel",
#                    "ignoreFailures": true
#                }
#            ]
#        },

# $1 is "--interpreter=mi", inserted by vscode
# $2 is "gdb" set with miDebuggerArgs
# $3 is "${workspaceFolder}/libtool" set with miDebuggerArgs
# $4 is "src/myprogram" set with miDebuggerArgs
# $5 is "--tty=/dev/pts/19", inserted by vscode

# LD_LIBRARY_PATH is set here but shouldn't be. Instead, libtool should
# add -rpath to the link command, so that --mode=execute runs the program
# with the correct LD_LIBRARY_PATH.
# I'm no libtool expert.
# If you know how to get libtool to add -rpath at link time, please let me know.
build_dir=$(dirname "$3")
export LD_LIBRARY_PATH=${build_dir}/lib/.libs/:$LD_LIBRARY_PATH

# Reorder the args and use solib-absolute-prefix to prevent GDB from seaching
# the / prefix first for shared libs.
# gdb will search the shared libs in the following locations, in this order:
#   1. additionalSOLibSearchPath,
#   2. $LD_LIBRARY_PATH
#   3. system linker's default search path (/lib:/usr/lib:...)
# This ensures the program runs with its own shared libs even if a copy could
# be found installed in /usr/lib/ or other system location.

[ -x "$3" ] && "$3" --mode=execute "$2" \
     -ex "set solib-absolute-prefix /dev/null" "$1" "$5" "$4"

The script should be copied into the project's root directory. The LD_LIBRARY_PATH setting should not be necessary, is a hack I don't know how to eliminate. Without it, the executable run by gdb is linked against the system libraries, rather than the staged libraries. If you know how to remove it, I'd appreciate a comment below.

The vscode launch.json will need a configuration similar to the following, as documented in the wrapper script:

        {
            "name": "(gdb) Launch",
            "type": "cppdbg",
            "request": "launch",
            "program": "src/.libs/myprogram",
//                     ^^^^^^^^^^^^^^^^^^^^^
            "additionalSOLibSearchPath": "lib/.libs/",
//                                       ^^^^^^^^^^^^
            "args": ["-4", "-myprogram-args"],
            "stopAtEntry": true,
            "cwd": "${workspaceFolder}",
            "externalConsole": false,
            "miDebuggerPath": "./vscode-lt-launch.sh",
//                            ^^^^^^^^^^^^^^^^^^^^^^^
            "miDebuggerArgs": "gdb ${workspaceFolder}/libtool src/myprogram",
//                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            "MIMode": "gdb",
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                }
            ]
        },

If libtool is needed to build and test your app before installation, it's needed to run the app in its uninstalled state.

Is there an advantage to using the libtool wrapper? This question is a bit like asking if there is an advantage to using a compiler to build your app, or a debugger to debug it.

Sometimes people don't know what function libtool has, and add it to the project "just in case", but without using its functions, so it's not uncommon for a project built with libtool to link and execute just fine by calling the binary directly (src/.lib/myprogram or such), especially for small projects without dependencies.

If the program depends on other libraries which are installed in a staging area, libtool will ensure that it runs against the staging area libs, even if libs with the same name are installed on the system. Eg. if you are testing changes to a common system library like libc or libresolv or such, you'd be staging your built libc/libresolv, then build/test the application against the staged lib, instead of the system-installed lib. You'd only install your improved libc once you're done testing it in isolation. Until then, the wrapper that libtool generates (eg, "src/myprogram") contains the right linking info so it can execute against the staged libraries.

libtool is essential to running the debugged app in the staging environment it was built against, but if the program is simple and only depends on libraries which are already installed on the system, libtool may not be needed in the project to begin with.