zipline-trader Data ingestion: Issue with datetime object

Question

The assumption here is that by multiplying 1e9, it will add milliseconds to the datetime objects (I might be wrong). Please look at the code below:

print(df.index)

Index(['2022-04-04T04:00:00Z', '2022-04-05T04:00:00Z'], dtype='object', name='time')

df.index = pd.to_datetime(df.index)
df.index = pd.to_datetime((df.index * 1e9).astype('int64'), utc = True).tz_convert(NY)

File "/opt/anaconda3/envs/python3-6-13/lib/python3.6/site-packages/alpaca_trade_api/entity.py", line 137, in df (df.index * 1e9).astype('int64'), utc = True).tz_convert(NY) File "/opt/anaconda3/envs/python3-6-13/lib/python3.6/site-packages/pandas/core/ops/invalid.py", line 53, in invalid_op raise TypeError(f"cannot perform {name} with this index type: {typ}") TypeError: cannot perform mul with this index type: Index

So it does not allow multiplication with index type. So I do this:

   df.index = pd.to_datetime((df.index.astype(np.int64) * 1e9).astype('int64'), utc = True).tz_convert(NY) 

But the datetime outputs are NaT

DatetimeIndex(['NaT', 'NaT'], dtype='datetime64[ns, America/New_York]', name='time', freq=None)

This bug is found in entity.py. Does anyone know how to fix this?

Answer 1

I can't follow your code because I get an error when I try to do df.index.astype(np.int64) "ValueError: invalid literal for int() with base 10"

here's an alternative approach that uses pd.to_datetime

import pandas as pd

df = pd.DataFrame(index=['2022-04-04T04:00:00Z', '2022-04-05T04:00:00Z'])
df.index.name = 'time'
print(df.index)
#Index(['2022-04-04T04:00:00Z', '2022-04-05T04:00:00Z'], dtype='object', name='time')

df.index = pd.to_datetime(df.index).tz_convert('America/New_York')

print(df.index)
#DatetimeIndex(['2022-04-04 00:00:00-04:00', '2022-04-05 00:00:00-04:00'], dtype='datetime64[ns, America/New_York]', name='time', freq=None)