Loading dataframe from 1.1.5 fails on pandas 1.0.1

Question

I have a dataframe saved to a pickle (with a bunch of other stuff, as a dictionary). It was saved when using pandas version 1.1.5.

I'm trying to open it with version 1.0.1 but get the following error

File "<stdin>", line 1, in <module>
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/generic.py", line 5272, in __getattr__
    if self._info_axis._can_hold_identifiers_and_holds_name(name):
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/generic.py", line 5272, in __getattr__
    if self._info_axis._can_hold_identifiers_and_holds_name(name):
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/generic.py", line 5272, in __getattr__
    if self._info_axis._can_hold_identifiers_and_holds_name(name):
  [Previous line repeated 493 more times]
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/generic.py", line 493, in _info_axis
    return getattr(self, self._info_axis_name)
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/generic.py", line 5270, in __getattr__
    return object.__getattribute__(self, name)
  File "pandas/_libs/properties.pyx", line 63, in pandas._libs.properties.AxisProperty.__get__
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/generic.py", line 5270, in __getattr__
    return object.__getattribute__(self, name)
RecursionError: maximum recursion depth exceeded while calling a Python object

Is there any way to overcome this error? I can save the dataframe again, but can't downgrade or upgrade versions on both computers.

Thanks

Answer 1

I was unable to replicate your exact error but I was able to get another error when I tried to read a pickle file created by pandas v1.1.5 using pandas v1.0.1. I was able to workaround the issue by saving the file using the feather format. Sample code:-

In [23]: cake # in version v1.1.5
Out[23]: 
     replicate recipe  temperature  angle  temp
1            1      A          175     42   175
2            1      A          185     46   185
3            1      A          195     47   195
4            1      A          205     39   205
5            1      A          215     53   215
..         ...    ...          ...    ...   ...
266         15      C          185     28   185
267         15      C          195     25   195
268         15      C          205     25   205
269         15      C          215     31   215
270         15      C          225     25   225

In [24]: cake.reset_index().to_feather("cake.feather")

Reading the file in v1.0.1:

In [15]: cake = pd.read_feather("cake.feather")
In [16]: cake
Out[16]: 
     index  replicate recipe  temperature  angle  temp
0        1          1      A          175     42   175
1        2          1      A          185     46   185
2        3          1      A          195     47   195
3        4          1      A          205     39   205
4        5          1      A          215     53   215
..     ...        ...    ...          ...    ...   ...
265    266         15      C          185     28   185
266    267         15      C          195     25   195
267    268         15      C          205     25   205
268    269         15      C          215     31   215
269    270         15      C          225     25   225

In [17]: pd.__version__
Out[17]: '1.0.1'

In [18]: cake.set_index('index') # To set the index

The drawbacks with this approach are that there's an additional dependency on pyarrow and saving in the feather binary format requires that you reset the index as I've done above.