Matplotlib and mpi4py: Tclerror: no display name and no $DISPLAY

Question

I post here a runnable example of a previous Python script, following the suggestion of ngoldbaum. At the end I would like to have root to execute a plot (from package scitools, which invokes matplotlib), but it runs into error and does not produce any output.

import matplotlib
matplotlib.use('Agg')
import sys
import numpy as np
import yt
from scitools.all import *
from scitools.std import *
from matplotlib import rc
from mpi4py import MPI

yt.enable_parallelism()


comm = MPI.COMM_WORLD
rank = comm.Get_rank()
nprocs = comm.Get_size()

print "rank, nprocs", rank, nprocs

# 
out_tmax = []
out_zmax = []
n_el = np.zeros(nprocs, dtype=int)

xmin = 0.
xmax = 2.2
deltax = (xmax - xmin)/nprocs

xmin_loc = xmin + rank*deltax
xmax_loc = xmin_loc + deltax

n_el[rank] = 10*(1 + rank)
if rank == 0:
    out_tmax = np.linspace(xmin_loc,xmax_loc,n_el[rank])
    out_zmax = sin(out_tmax)
    print "ok", out_tmax, out_zmax

# Plot
if yt.is_root():
#if rank == 0:
    rc('font',**{'family':'serif','serif':['Times']})
## for Palatino and other serif fonts use:
#    rc('font',**{'family':'serif','serif':['Palatino']})
    rc('text', usetex=True)
    rc('legend',fontsize=12)
    rc('axes',labelsize=14, titlesize=18)
    rc('xtick', labelsize=14)
    rc('ytick', labelsize=14)
    plot(out_tmax, out_zmax, 'b', fontname='Times new Roman', xlabel=(r'Time (Myr)'), ylabel=(r'\mathrm{z_{HS}}'), hardcopy='fig1.png')

It looks like the root process did not inherit the "use: Agg" choice made at the beginning (I do not get any *png output as specified in plot(). I submit it to the PBS batch system using s cript:

#!/bin/bash
## Name of the job
#PBS -N test_mpi4py_matlib

#PBS -l select=1:ncpus=4:mpiprocs=4:mem=24gb
#PBS -l place=scatter

#PBS -e test_gatherv_print.err
#PBS -o test_gatherv_print.out
#PBS -l walltime=00:20:00
....
mpirun python2.7 test_gatherv_print.py

The first two lines of this script should prevent plot (based on matplotlib) to look for the DISPLAY environment variable.

This script fails with the following error:

  File "test_gatherv_print.py", line 49, in <module>
    plot(out_tmax, out_zmax, 'b', fontname='Times new Roman', xlabel=(r'Time (Myr)'), ylabel=(r'\mathrm{z_{H
S}}'), hardcopy='fig1.png')
  File "/home/van/software/python/scitools/lib/python2.7/site-packages/scitools/easyviz/common.py", line 313
9, in plot
    self.setp(**kwargs)
  File "/home/van/software/python/scitools/lib/python2.7/site-packages/scitools/easyviz/common.py", line 194
1, in setp
    self.hardcopy(kwargs['hardcopy'])
  File "/home/van/software/python/scitools/lib/python2.7/site-packages/scitools/easyviz/matplotlib_.py", lin
e 1065, in hardcopy
    self._replot()
  File "/home/van/software/python/scitools/lib/python2.7/site-packages/scitools/easyviz/matplotlib_.py", lin
e 944, in _replot
    self._g.figure(self.getp('curfig'))
  File "/opt/Python-2.7.9/lib/python2.7/site-packages/matplotlib/pyplot.py", line 533, in figure
    **kwargs)
  File "/opt/Python-2.7.9/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 161, in new_figure_
manager
    return cls.new_figure_manager_given_figure(num, fig)
  File "/opt/Python-2.7.9/lib/python2.7/site-packages/matplotlib/backends/_backend_tk.py", line 1046, in new
_figure_manager_given_figure
    window = Tk.Tk(className="matplotlib")
  File "/opt/Python-2.7.9/lib/python2.7/lib-tk/Tkinter.py", line 1810, in __init__
    self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
P000 yt : [ERROR    ] 2020-05-13 16:28:32,964 TclError: no display name and no $DISPLAY environment variable
P000 yt : [ERROR    ] 2020-05-13 16:28:32,965 Error occured on rank 0.

although I specified the 'Agg' output in matplotlib.use() above.

Answer 1

Thank you for your suggestions. I had already tried all what you suggest: 1) matplotlibrc was configured (backend: Agg) both in the local directory and from under .config; 2) used matplotlib commands instead of the "plot" from scitools. Concerning the latter, it is a package of utilities developed by P. Langtangen; in particular I used here the easyviz subpackage (see: https://hplgit.github.io/scitools/doc/api/html/easyviz.html). However even when I DO NOT use scitools.easyviz and instead use directly matplotlib I am unable to produce a .png output from within a MPI spawned root process. I substituted the following lines at the end:

...
    plt.plot(out_tmax, out_zmax)
    plt.savefig('fig1.png')

and now I get the following error:

P000 yt : [ERROR    ] 2020-05-14 10:32:14,200 OSError: [Errno 2] No such file or directory: 'latex'
P000 yt : [ERROR    ] 2020-05-14 10:32:14,200 Error occured on rank 0.

Note that when I run interactively the script it correctly produces the fig1.png output. As you may see the problem has now shifted: root does not read some environment variables. Note that I submit the job to PBS batch system. The last script was like that:

#!/bin/tcsh
## Name of the job
#PBS -N test_mpi4py_matlib

## Number of nodes (in this case I require 8 nodes with 1 CPU each)
## The total number of nodes passed to mpirun will be nodes*ppn 
##PBS -l nodes=5:ppn=16
##PBS -l nodes=2:ppn=8
#PBS -l select=1:ncpus=4:mpiprocs=4:mem=24gb
#PBS -l place=scatter
#PBS -V 
# inherits all environment variables, including DISPLAY

#! Name of output files for std output and error;
#! if non specified defaults are <job-name>.o<job number> and <job-name>.e<job-number>
#
#PBS -e test_gatherv_print.err
#PBS -o test_gatherv_print.out
#PBS -l walltime=00:20:00
##PBS -q high


#comandi
#printenv LD_LIBRARY_PATH
printenv DISPLAY

cd /home/van/projects/jet_ism/turbulence/python

date
#echo 'Hello world'
/opt/openmpi_icc/bin/mpirun python2.7 test_gatherv_print.py

Here I use tcsh, BUT before I used also ksh.