pyarrow Table adds column '__index_level_0__'

68 Views Asked by At

If I create a pandas df and convert that to a pyarrow Table, I got an additional column 'index_level_0'.

How do I get rid of it?

from pyarrow import Table
import pandas as pd

df_empty = pd.DataFrame(columns=["a", "b", "c",])
df_empty = df_empty.astype(
    {"a": "int64", "b": "datetime64[ns]", "c": "int64"}
)
df = Table.from_pandas(df_empty)

enter image description here

2

There are 2 best solutions below

0
daniel guo On

I got it to work with preserve_index=False

return Table.from_pandas(df_empty, preserve_index=False)
0
Mr. Irrelevant On

You could try out "preserve_index=False"

from pyarrow import Table
import pandas as pd

df_empty = pd.DataFrame(columns=["a", "b", "c",])
df_empty = df_empty.astype(
  {"a": "int64", "b": "datetime64[ns]", "c": "int64"}
)
df = Table.from_pandas(df_empty, preserve_index = False)