How to forecast unknown future target values with gluonts DeepAR?

Question

How to forecast unknown future target values with gluonts DeepAR? I have a time series from 1995-01-01 to 2021-10-01. Monthly frequency data. How to forecast values for the future (next 3 months): 2021-11-01 to 2022-01-01? Note that I don't have the target values for 2021-11-01, 2021-12-01 and 2022-01-01. Many thanks!

from gluonts.model.deepar import DeepAREstimator
from gluonts.mx import Trainer 
import numpy as np
import mxnet as mx

np.random.seed(7)
mx.random.seed(7)

estimator = DeepAREstimator(
    prediction_length=12
    , context_length=120
    , freq='M'
    , trainer=Trainer(        
        epochs=5
        , learning_rate=1e-03
        , num_batches_per_epoch=50))

predictor = estimator.train(training_data=df_train)

# Forecasting
predictions = predictor.predict(df_test)
predictions = list(predictions)[0]
predictions = predictions.quantile(0.5)

print(predictions)
[163842.34  152805.08  161326.3   176823.97  127003.79  126937.78
 139575.2   117121.67  115754.67  139211.28  122623.586 120102.65 ]

As I understood, the predictions values are not for "2021-11-01", "2021-12-01" and "2022-01-01". How do I know to which months this values refer to? How to forecast values for the next 3 months: "2021-11-01", "2021-12-01" and "2022-01-01"?

Take a look at this code. It comes from "Advanced Forecasting with Python". https://github.com/Apress/advanced-forecasting-python/blob/main/Chapter%2020%20-%20Amazon's%20DeepAR.ipynb

It does not seem to forecast unknown future values, once it compares the last 28 values of test_ds (Listing 20-5. R2 score and prediction graph) with the predictions made over this same dataset test_ds (Listing 20-4. Prediction)

How do I forecast unknown future values?

Many thanks!

Data source

https://www.kaggle.com/c/recruit-restaurant-visitor-forecasting

# Listing 20-1. Importing the data
import pandas as pd
y = pd.read_csv('air_visit_data.csv.zip')
y = y.pivot(index='visit_date', columns='air_store_id')['visitors']
y = y.fillna(0)
y = pd.DataFrame(y.sum(axis=1))

y = y.reset_index(drop=False)
y.columns = ['date', 'y']


# Listing 20-2. Preparing the data format requered by the gluonts library
from gluonts.dataset.common import ListDataset
start = pd.Timestamp("01-01-2016", freq="H")
# train dataset: cut the last window of length "prediction_length", add "target" and "start" fields
train_ds = ListDataset([{'target': y.loc[:450,'y'], 'start': start}], freq='H')
# test dataset: use the whole dataset, add "target" and "start" fields
test_ds = ListDataset([{'target': y['y'], 'start': start}],freq='H')


# Listing 20-3. Fitting the default DeepAR model
from gluonts.model.deepar import DeepAREstimator
from gluonts.trainer import Trainer
import mxnet as mx
import numpy as np

np.random.seed(7)
mx.random.seed(7)

estimator = DeepAREstimator(
    prediction_length=28,
    context_length=100,
    freq=’H’,
    trainer=Trainer(ctx="gpu", # remove if running on windows
                    epochs=5,
                    learning_rate=1e-3,
                    num_batches_per_epoch=100
                   )
)

predictor = estimator.train(train_ds)



# Listing 20-4. Prediction
predictions = predictor.predict(test_ds)
predictions = list(predictions)[0]
predictions = predictions.quantile(0.5)



# Listing 20-5. R2 score and prediction graph
from sklearn.metrics import r2_score
print(r2_score( list(test_ds)[0]['target'][-28:], predictions))

import matplotlib.pyplot as plt
plt.plot(predictions)
plt.plot(list(test_ds)[0]['target'][-28:])
plt.legend(['predictions', 'actuals'])
plt.show()

Answer 1

In your case the context length is 120 and prediction length is 12 so the model will look behind 120 data points to predict 12 future data points

The recommendation is to reduce the context to may be 10 and include the data from past 10 months in the df_test table

you can get the start of the forecast using

list(predictor.predict(df_test))[0].start_date

based on this create a future table of 12 dates(as 12 is the prediction length)