Backtrader - IndexError: list index out of range

Question

I'm trying to load a custom data feed from a csv file that I've generated called "impact.csv" through the script below "backtrader_impact.py".

However, at "cerebro.adddata(data)", I receive an IndexError: list index out of range.

  File "C:\...\backtrader\feeds\csvgeneric.py", line 151, in _loadline
    csvfield = linetokens[csvidx]
IndexError: list index out of range

backtrader_impact.py:

import backtrader as bt

class ImpactCSVData(bt.feeds.GenericCSVData):
    lines = ('impact_score',)
    
    params = (
        ('dtformat', '%Y/%m/%d %H:%M:%S'),  # Datetime format
        ('datetime', 0),  # Index of the datetime column in the CSV
        ('impact_score', 1),  # Index of the impact_score column in the CSV
        ('timeframe', bt.TimeFrame.Minutes),  # Timeframe (adjust accordingly)
    )

data = ImpactCSVData(
    dataname='alpaca_backtrader/impact.csv',  # Path to your CSV file
)

cerebro = bt.Cerebro()
cerebro.adddata(data)

# cerebro.addstrategy(YourStrategy)

cerebro.run()

# cerebro.plot()

impact.csv:

datetime,impact_score
2023/01/13 00:00:00,96
2023/01/13 00:15:00,75
2023/01/13 00:30:00,58

I've tried a bit of debugging by adding some print statements in backtrader\feeds\csvgeneric.py (in backtrader package itself):

        for linefield in (x for x in self.getlinealiases() if x != 'datetime'):
            # Get the index created from the passed params
            csvidx = getattr(self.params, linefield)

            if csvidx is None or csvidx < 0:
                # the field will not be present, assignt the "nullvalue"
                print(f"Processing line tokens: {linetokens}, with csvidx: {csvidx}")
                csvfield = self.p.nullvalue
            else:
                # get it from the token
                # FIXME: failing here. trying to access index 4 of list of 2 items
                print(f"Processing line tokens: {linetokens}, with csvidx: {csvidx}")
                csvfield = linetokens[csvidx]

            if csvfield == '':
                # if empty ... assign the "nullvalue"
                print(f"Processing line tokens: {linetokens}, with csvidx: {csvidx}")
                csvfield = self.p.nullvalue

What is printed:
Processing line tokens: ['2023/01/13 00:00:00', '96'], with csvidx: 4

For some reason it is trying to access index 4 of linetokens that is just ['2023/01/13 00:00:00', '96'] with a length of 2.

Would appreciate your help! Thank you.

Answer 1

I had the same issue when I want to import custom csv file to the datafeed. The problem with GenericCSVData() class is that it will look for Date, Open, High, Low, Close, Volume, Open_interest columns regardless of your file.

The first problem with your code is that the csv you input only has 2 columns but the class is looking for 6 and when it looks for data.csv[3] it is out of range.

The second problem is that you didn't input the value of the line impact_score. You should adjust this lines = ('impact_score', 7) and then in the custom data feed you should clarify the index of your data

data = ImpactCSVData(
    dataname='alpaca_backtrader/impact.csv',
    datetime=0,
    open=1,
    high=2,
    low=3,
    close=4,
    volume=5,
    openinterest=6,
    impact_score=7
)

After this you can fill in your CSV file with dummy data for OHLCV and then you can access the impact score like you mentioned.