Using PETSc on makefile within my user-defined makefile

Question

I am new in PETSc. I have a big c++ code and I want to add PETSc to some of the files that I already have, so I have to change my makefile in a way that it can compile PETSc as well.

Is it possible to have two different makefiles and then call PETSc makefile within my own makefile? if so, how can I do this?

Does anyone have any experience in linking PETSc to their own code?

By the way, I am using Linux as my operating system.

Answer 1

edit: Though this is an old post, I am sure there are still people struggling with this.

Furthermore, things apparently changed slightly for petsc 3.6.x (and slepc 3.6.x).

I currently use the following lines in my ubuntu 14.04 LTS makefile for F90 files (using both petsc 3.6.1 and slepc 3.6.0):

# PETSC and SLEPC directories
PETSC_DIR = /opt/petsc-3.6.1
SLEPC_DIR = /opt/slepc-3.6.0
include  $(PETSC_DIR)/lib/petsc/conf/variables
include  $(SLEPC_DIR)/lib/slepc/conf/slepc_variables

Using this I can construct the

# Compiler command
COMPILE = $(COMP_DIR) $(COMP_FLAGS) $(PETSC_FC_INCLUDES) $(SLEPC_INCLUDE)

where COMP_DIR has to be set manually (e.g. COMP_DIR = /usr/bin/mpif90 or COMP_DIR = /usr/bin/gfortran) and COMP_FLAGS are additional flags (e.g. '-g O0'), as well as the

# Link command
LINK = $(LINK_DIR) &(LINK_FLAGS)

where again LINK_DIR has to be set manually (e.g. /usr/bin/g++) and LINK_FLAGS contains additional flags (e.g. -fPIC).

These can then be used to create rules to compile the F90 files (I am sure C is pretty similar):

%.o : %.f90
$(COMPILE) -c $<

and the main program:

main:   $(ObjectFiles) main.o
$(LINK) -o $@ $(ObjectFiles) main.o $(LINK_LIB) $(PETSC_LIB) $(SLEPC_LIB)

where ObjectFiles contains a list of all the files in the project and LINK_LIB corresponds to other links (e.g. -lgfortran).

This works fine for me, yet suggestions for improvements are always welcome.

original post: Instead of the hack described by Divakar, you can easily find out the link flags and the include directories for compilation by running

make getlinklibs
make getincludedirs

in the main petsc directory, as described here...

Answer 2

I am not too familiar with Makefiles, so I would just list out the “hack” method. We will look into the “hack” method in this text later on. I have a Makefile and a sample source code, ex1.cpp that uses few PETSc arrays, vectors, functions along with my own regular C/C++ array that does data exchange with PETSc array and vectors. This could be thought of as a miniature version of your case.

My Makefile –

PETSC_DIR=/usr/local/petsc

include ${PETSC_DIR}/conf/variables
include ${PETSC_DIR}/conf/rules
include ${PETSC_DIR}/conf/test

CLINKER=g++

ex1 : ex1.o chkopts
    ${CLINKER} -w -o ex1 ex1.o ${PETSC_LIB}
    ${RM} ex1.o 
    ./ex1

Of course you need to edit PETSC_DIR to the PETSc directory location on your system. Typing “make ex1” would compile and link the source code to create an executable and execute it.

After I do “make ex1” on my system, the two process outputs of compilation and linking are shown, that are listed here as follows:

Compilation -

/usr/local/petsc/arch-linux2-c-debug/bin/mpicc -o ex1.o -c -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -fno-inline -O0 -I/usr/local/petsc/include -I/usr/local/petsc/arch-linux2-c-debug/include ex1.cpp

Linking -

g++ -w -o ex1 ex1.o -Wl,-rpath,/usr/local/petsc/arch-linux2-c-debug/lib -L/usr/local/petsc/arch-linux2-c-debug/lib -lpetsc -Wl,-rpath,/usr/local/petsc/arch-linux2-c-debug/lib -lflapack -lfblas -lX11 -lpthread -lm -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/4.6 -L/usr/lib/gcc/x86_64-linux-gnu/4.6 -Wl,-rpath,/usr/lib/x86_64-linux-gnu -L/usr/lib/x86_64-linux-gnu -Wl,-rpath,/lib/x86_64-linux-gnu -L/lib/x86_64-linux-gnu -lmpichf90 -lgfortran -lm -lgfortran -lm -lquadmath -lm -lmpichcxx -lstdc++ -ldl -lmpich -lopa -lmpl -lrt -lpthread -lgcc_s –ldl

So the “hack” trick is that you run the Makefile and separate the compilation and linking process outputs with this PETSc case. You do the same with your original source code that is PETSc-free and note down compilation and linking process outputs with it.

Let’s suppose with PETSc-free version, the compilation process output is g++ -o ex1.o –I/random_path ex1.cpp and linking process output is g++ -w -o ex1 ex1.o –llib1 –L/random_lib2.

Next step is to merge the compilation paths for PETSc code and PETSc-free code and same with linking. Thus, the modified compilation and linking processes would be:

Modified Compilation -

/usr/local/petsc/arch-linux2-c-debug/bin/mpicc -o ex1.o -c -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g3 -fno-inline -O0 -I/usr/local/petsc/include -I/usr/local/petsc/arch-linux2-c-debug/include –I/random_path ex1.cpp

Modified Linking –

g++ -w -o ex1 ex1.o -Wl,-rpath,/usr/local/petsc/arch-linux2-c-debug/lib -L/usr/local/petsc/arch-linux2-c-debug/lib -lpetsc -Wl,-rpath,/usr/local/petsc/arch-linux2-c-debug/lib -lflapack -lfblas -lX11 -lpthread -lm -Wl,-rpath,/usr/lib/gcc/x86_64-linux-gnu/4.6 -L/usr/lib/gcc/x86_64-linux-gnu/4.6 -Wl,-rpath,/usr/lib/x86_64-linux-gnu -L/usr/lib/x86_64-linux-gnu -Wl,-rpath,/lib/x86_64-linux-gnu -L/lib/x86_64-linux-gnu -lmpichf90 -lgfortran -lm -lgfortran -lm -lquadmath -lm -lmpichcxx -lstdc++ -ldl -lmpich -lopa -lmpl -lrt -lpthread -lgcc_s –ldl –llib1 –L/random_lib2

You may type the modified commands directly onto the terminal or make a BASH script to run them.

The PETSc example code that calculates reciprocal of the numbers in an array is listed below for the sake of reference:

// EX1.CPP
#include <petscvec.h>
#include <petscmat.h>
#include <petscksp.h>

Vec Arr2Vec(double *arr2, int SIZE);

// MAIN FUNCTION
int main(int argc,char **argv)
{
    // Initialize PetSc
    PetscInitialize(&argc,&argv,(char*)0,"Testing a program!");

    // Initialize parameters
    int SIZE = 3;
    PetscErrorCode ierr;

    // **** Create a regular arary and set it with random numbers
    double  * arr2;
    arr2 = new double [SIZE];

    arr2[0] = 0.1;
    arr2[1] = 0.4;
    arr2[2] = 0.2;

    // Convert regular arary to PETSc vector [Note that this must do the same effect as the two-step process of conversion from regular array to PETSc arary and that to PETSc vector as listed above]
    Vec x = Arr2Vec(arr2, SIZE);

    printf("Reciprocal Vector : \n"); VecReciprocal(x);
    VecView(x,PETSC_VIEWER_STDOUT_WORLD);

    //Cleanup
    ierr = VecDestroy(&x);
    CHKERRQ(ierr);
    PetscFinalize();

    return 0;
}

Vec Arr2Vec(double *arr2, int SIZE)
{
  PetscScalar *array1;
  PetscMalloc(SIZE*sizeof(PetscScalar),&array1);

  for(int i=0;i<SIZE;i++)
    array1[i]=arr2[i];

    // Setup vector
  Vec x;
  VecCreate(PETSC_COMM_WORLD,&x);
  VecSetSizes(x,PETSC_DECIDE,SIZE);
  VecSetFromOptions(x);

  // Place PetSc array as Vector
  VecPlaceArray(x,array1);

  return x;

}

Answer 3

I have met with the same problem, and for that I made a simple but comprehensive makefile (to be used under Linux OS). Suppose that the code is stored in ex_1.cpp. PETSc is assumed to be installed in /usr/local/petsc/petsc-3.7.4. One can find the proper environmental variables in configure.log file after installation of PETSc.

One can then use the following makefile to compile and link the C++ code:

# PETSC 3.7.4 Makefile (Linux)
#----------------------------------------

# NOTE: look up `configure.log` in PETSc folder to obtain
# the proper environmental parametres
PETSC_DIR=/usr/local/petsc/petsc-3.7.4
PETSC_ARCH=arch-linux2-c-debug
PETSC_CONFIGDIR=${PETSC_DIR}/lib/petsc/

include ${PETSC_CONFIGDIR}/conf/variables
include ${PETSC_CONFIGDIR}/conf/rules
include ${PETSC_CONFIGDIR}/conf/test

# compile and link options
LOCDIR= ./
DIRS =
CXX_STD = -std=c++11
CXX_CFLAGS = ${CXX_STD} ${CXX_FLAGS} ${PETSC_CCPPFLAGS} 
LIBS = ${PETSC_LIB}
CXX_LFLAGS = ${CXX_STD}

#OBJS = $(SRC1:.cpp=.o)
#----------------------------------------

.PHONY: default allclean run_1

default: chkopts
    @echo "--- PETSC CONFIGURATION -----------------"
    @if [ "${CUDAC}" != "" ]; then \
        echo "Using CUDA compile: ${PETSC_CUCOMPILE}";\
    fi
    @echo "Using C/C++ linker: ${PCC_LINKER}"
    @echo "Using C/C++ flags: ${PCC_LINKER_FLAGS}"
    @echo "Using C++ flags: ${CXX_FLAGS}"
    @echo "-----------------------------------------"
    @echo "Using libraries: ${PETSC_LIB}"
    @echo "-----------------------------------------"
    @echo "Using mpiexec: ${MPIEXEC}"
    @echo "========================================="

ex_1: default ex_1.o 
    @echo "---- LINK -----"
    ${CXX} -w -o ex_1.out ex_1.o ${LIBS} ${CXX_LFLAGS}
    -${RM} ex_1.o
    @echo "==============="

ex_1.o:
    @echo "--- COMPILE ---"
    ${CXX} -o ex_1.o -c ex_1.cpp ${CXX_CFLAGS}
    @echo "==============="

run_1:
    @echo "==============="
    @echo "--- EXECUTE ---"
    @echo "starting on `hostname` at `date`"
    @echo "machine characteristics: `uname -a`"
    @echo "==============="
    ${MPIEXEC} -n 1 ./ex_1.out #-info
    @echo "==============="

 allclean: clean
    -@${RM} *.out

Note that chkopts and clean rules are already defined by PETSc. The executable file will be stored as ex_1.out. To compile and link the code, use make ex_1 and to run the executable file, use make run_1.

Answer 4

This is an old post, however I am sure there are still people struggling with this

I didn't tried in C/C++ but in Fortran, adding the correct folders and libraries, in the compiler/linker commands, worked for me:

-I${PETSC_DIR}/include
-L${PETSC_DIR}/lib -lpetscsys -lpetscXXX

without the need of including variables/rules/test file, that broke my makefile.

Hope it helps...