I'm trying to launch an app running under Panel in a docker container, it loads a netcdf file and make some plots, it is supposed to look like this:
The application code is available here: https://github.com/Paleoclim-CNRS/netcdf_editor_app/blob/main/Single_Page_WebApp/app.ipynb (Do not take in account the requirements.txt in the GitHub repo which has pinned older versions for all the packages above and the app is not working either with these)
Issue
When starting the container, the app launches and allow to choose a netcdf file but once the file is selected, nothing happens and an error is triggered:
b'\x00\x00\x00\x00\xc0i\x00\x00\x00\x00\x00\x00\xc0i\x00\x00'
...
b'\x00\x00\x00\x00\xc0\xaa\xe0\x00\x00\x00\x00\x00\xc0\xaa\xe0\x00'
...
b'\x00\x00\x00\x00\xc0\xaa\xe0\x00\x00\x00\x00\x00\xc0\xaa\xe0\x00'
b'\x80\x00\x00\x00\xc0\xaaF4\x80\x00\x00\x00\xc0\xaaF4'
...
...
b'\x80\x00\x00\x00\xc0\xaaF4\x80\x00\x00\x00\xc0\xaaF4'
b'\x80\x00\x00\x00'}
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/xarray/core/dataset.py", line 1348, in _construct_dataarray
variable = self._variables[name]
KeyError: None
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/panel/reactive.py", line 381, in _process_events
self.param.update(**self_events)
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 1902, in update
self_._batch_call_watchers()
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 2063, in _batch_call_watchers
self_._execute_watcher(watcher, events)
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 2025, in _execute_watcher
watcher.fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 669, in caller
return function()
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 407, in _depends
return func(*args, **kw)
File "/usr/src/app/app.ipynb", line 209, in _parse_file_input
" else:\n",
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 369, in _f
return f(self, obj, val)
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 1252, in __set__
obj.param._call_watcher(watcher, event)
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 2043, in _call_watcher
self_._execute_watcher(watcher, (event,))
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 2025, in _execute_watcher
watcher.fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 669, in caller
return function()
File "/usr/local/lib/python3.8/site-packages/param/parameterized.py", line 407, in _depends
return func(*args, **kw)
File "/usr/src/app/app.ipynb", line 651, in get_plots
" passage_problems = hv.DynamicMap(self.load_passage_problems).opts(\n",
File "/usr/local/lib/python3.8/site-packages/xarray/core/dataset.py", line 1439, in __getitem__
return self._construct_dataarray(key)
File "/usr/local/lib/python3.8/site-packages/xarray/core/dataset.py", line 1350, in _construct_dataarray
_, name, variable = _get_virtual_variable(self._variables, name, self.dims)
File "/usr/local/lib/python3.8/site-packages/xarray/core/dataset.py", line 182, in _get_virtual_variable
raise KeyError(key)
KeyError: None
I don't have much experience with all of theses packages and I have no idea where the issue could come from. The combinations of versions for Panel/bokeh/holoviews/xarray I tried were not successful so far.
I get a bunch of binary strings at the top of the error b'\x80\x00\x00\x00\xc0\xaaF4\x80\x00\x00\x00\xc0\xaaF4', so I was wondering if the netcdf file was read properly (h5netcdf/nettcdf4 packages ?) hence the error with xarray (?)
Or it could be a package version which could break up everything...
Do you have any leads on what I should look at to solve this ?
packages used
The python packages used for this:
matplotlib 3.7.1
xarray 2023.1.0
panel 1.2.0
bokeh 2.4.3
holoviews 1.16.2
datashader 0.15.0
hvplot 0.8.4
shapely 2.0.1
h5netcdf 1.1.0
netcdf4 1.6.3
scipy 1.10.1
requests 2.31.0
spatialpandas 0.4.8
scikit-image 0.21.0
jinja2 3.1.2
jupyter 1.0.0
Code
I'm not sure about what part of the script is getting into trouble but the app is defined through a class with the __init__ method:
def __init__(self, **params):
self.param.file.default = pn.widgets.FileInput(max_width=200)
self.param.ds.default = xr.Dataset()
self.param.loaded.default = False
super().__init__(**params)
self.apply.on_click(self._apply_values)
self.undo_button.on_click(self.undo)
self.redo_button.on_click(self.redo)
self.download_netcdf.callback = self._download_netcdf
self.download_script.callback = self._download_script
self.fill_depressions_button.on_click(self._fill_depressions_callback)
self.file_pane.append(self.file)
self._auto_update_cmap_min = True
self._auto_update_cmap_max = True
self.curvilinear_coordinates = None
self._undo_list = []
self._redo_list = []
self.colormap_min.param.watch(self._colormap_callback, 'value')
self.colormap_max.param.watch(self._colormap_callback, 'value')
self.colormap_range_slider.param.watch(self._colormap_callback, 'value')
And the loaded file is processed with this method:
@pn.depends("file.value", watch=True)
def _parse_file_input(self):
self.loaded = False
value = self.file.value
# We are dealing with a h5netcdf file ->
# The reader can't read bytes so we need to write it to a file like object
if value.startswith(b"\211HDF\r\n\032\n"):
value = io.BytesIO(value)
ds = xr.open_dataset(value)
self.curvilinear_coordinates = None
number_coordinates_in_system = len(list(ds.coords.variables.values())[0].dims)
# Standard Grid
if number_coordinates_in_system == 1:
pass
# Curvilinear coordinates
elif number_coordinates_in_system == 2:
dims = list(ds[list(ds.coords)[0]].dims)
# Store the true coordinates for export
self.curvilinear_coordinates = list(ds.coords)
# Add the dimension into the coordinates this results in an ij indexing
ds.coords[dims[0]] = ds[dims[0]]
ds.coords[dims[1]] = ds[dims[1]]
# Remove the curvilinear coordinates from the original coordinates
ds = ds.reset_coords()
else:
raise ValueError("Unknown number of Coordinates")
self.ds = ds
self.attribute.options = list(ds.keys())
self._original_ds = ds.copy(deep=True)
self.loaded = True
return True
Thank you for your help !