Trouble in running julia on cluster

Question

I am running some simulations on a SLURM cluster with julia. I am novice in julia and can't seem to figure out what is missing. I have three main codefiles "job.sh", "main.jl" and "run-experiments.jl" and a proejct.toml and Manifest.toml file in the same directory. In "Job.sh", I have the following:

#!/bin/bash

#SBATCH --job-name=RunDMRG
#SBATCH --nodes=4
#SBATCH --oversubscribe
#SBATCH --time=08:00:00
#SBATCH --output=output_%j.txt
#SBATCH --error=error_%j.txt
#SBATCH --account=myname

module load julia

julia -e 'import Pkg; Pkg.instantiate()'  # Install packages

julia main.jl 256 100

and the main.jl file contains :

using ITensors
using Plots
using Printf
using Statistics
using LaTeXStrings
using CSV

using Pkg; Pkg.instantiate()
include("run-experiments.jl")

disorder = [10.0^i for i in -4:1]
nsites = parse(Int, ARGS[1]) 
nrealizations = parse(Int, ARGS[2]) 
print("Starting...")

main(nsites, nrealizations, disorder)

where the function main() is contained in run-experiments.jl.

When I submit the job, I immediately get an error stating that "iTensor is not installed". More specifically, the error file says :

ERROR: LoadError: ArgumentError: Package ITensors not found in current path:
- Run `import Pkg; Pkg.add("ITensors")` to install the ITensors package.

Stacktrace:
 [1] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:893
in expression starting at /scratch/zt1/project/main.jl:1

But isn't that what Pkg.instantiate() is supposed to do? Would appreciate any help on how to resolve this and run a clean simulation. Thanks!

Answer 1

Julia uses the built-in environment and package system Pkg.jl. These environments are encoded in the Project.toml and Manifest.toml files, but in order to use them, you have to switch to / activate the environment that you want to use. The first time you do this on a new system, instantiate makes sure that all packages that are listed in the environment are installed. So, you don't necessarily always need the instantiate in your Slurm script or Julia script, but instead, your Julia script should start with

import Pkg
Pkg.activate(".") # or the actual path to the folder where your Enviromentment is

This is just a very short explaination, and if you are not familiar with Pkg and the environment system, you should really have a look at the documentation: https://pkgdocs.julialang.org/v1/getting-started/