TensorFlow RNN Execution Error: Incorrect rnn_mode, rnn_input_mode, and rnn_direction_mode Configuration

38 Views Asked by At
# Preparing the data
import os, pathlib, shutil, random
from tensorflow import keras
batch_size = 32
base_dir = pathlib.Path("aclImdb")
val_dir = base_dir / "val"
train_dir = base_dir / "train"
for category in ("neg", "pos"):
    os.makedirs(val_dir / category)


    files = os.listdir(train_dir / category)
    random.Random(1337).shuffle(files)
    num_val_samples = int(0.2 * len(files))
    val_files = files[-num_val_samples:]
    for fname in val_files:
        shutil.move(train_dir / category / fname,
                    val_dir / category / fname)

train_ds = keras.utils.text_dataset_from_directory(
    "aclImdb/train", batch_size=batch_size
)
val_ds = keras.utils.text_dataset_from_directory(
    "aclImdb/val", batch_size=batch_size
)
test_ds = keras.utils.text_dataset_from_directory(
    "aclImdb/test", batch_size=batch_size
)
text_only_train_ds = train_ds.map(lambda x, y: x)
from tensorflow.keras import layers
# Preparing integer sequence datasets
max_length = 600
max_tokens = 20000
text_vectorization = layers.TextVectorization(
    max_tokens=max_tokens,
    output_mode="int",
    output_sequence_length=max_length,
)
text_vectorization.adapt(text_only_train_ds)

int_train_ds = train_ds.map(
    lambda x, y: (text_vectorization(x), y),
    num_parallel_calls=4)
int_val_ds = val_ds.map(
    lambda x, y: (text_vectorization(x), y),
    num_parallel_calls=4)
int_test_ds = test_ds.map(
    lambda x, y: (text_vectorization(x), y),
    num_parallel_calls=4)
# A sequence model built on one-hot encoded vector sequences
import tensorflow as tf
inputs = keras.Input(shape=(None,), dtype="int64")
embedded = tf.one_hot(inputs, depth=max_tokens)
x = layers.Bidirectional(layers.LSTM(32))(embedded)
x = layers.Dropout(0.5)(x)
outputs = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(inputs, outputs)
model.compile(optimizer="rmsprop",
              loss="binary_crossentropy",
              metrics=["accuracy"])
model.summary()
callbacks = [
    keras.callbacks.ModelCheckpoint("one_hot_bidir_lstm.keras",
                                    save_best_only=True)
]
model.fit(int_train_ds, validation_data=int_val_ds, epochs=10, callbacks=callbacks)
model = keras.models.load_model("one_hot_bidir_lstm.keras")
print(f"Test acc: {model.evaluate(int_test_ds)[1]:.3f}")

above are my code for fit a bidirectional LSTM model for solving IMDB film comments emotional classification. When I tried to build a bidirectional LSTM model, the following error occurred:

--------------------------------------------
InternalError                             Traceback (most recent call last)
Cell In[5], line 18
13 model.summary()
14 callbacks = [
15     keras.callbacks.ModelCheckpoint("one_hot_bidir_lstm.keras",
16                                     save_best_only=True)
17]
--->  18 model.fit(int_train_ds, validation_data=int_val_ds, epochs=10, callbacks=callbacks)
19 model = keras.models.load_model("one_hot_bidir_lstm.keras")
20 print(f"Test acc: {model.evaluate(int_test_ds)[1]:.3f}")

File F:\Python-virtual-envir\keras-deeplearning-envir\lib\site-packages\keras\utils\traceback_utils.py:70,  in filter_traceback.< locals> .error_handler(*args, **kwargs)
67     filtered_tb = _process_traceback_frames(e.__traceback__)
68     # To get the full stack trace, call:
69     # `tf.debugging.disable_traceback_filtering()`
--->  70     raise e.with_traceback(filtered_tb) from None
71 finally:
72     del filtered_tb

File F:\Python-virtual-envir\keras-deeplearning-envir\lib\site-packages\tensorflow\python\eager\execute.py:54,  in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
52 try:
53   ctx.ensure_initialized()
--->  54   tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
55                                       inputs, attrs, num_outputs)
56 except core._NotOkStatusException as e:
57   if name is not None:

InternalError: Graph execution error:

Failed to call ThenRnnForward with model config: [rnn_mode, rnn_input_mode, rnn_direction_mode]:  2, 0, 0 , [num_layers, input_size, num_units, dir_count, max_seq_length, batch_size, cell_num_units]:  [1, 20000, 32, 1, 600, 32, 32]
[[{{node CudnnRNN}}]]
[[model/bidirectional/backward_lstm/PartitionedCall]] [Op:__inference_train_function_6753]
------------------------------------------

According to the analysis: In my model, I use the Bidirectional wrapper to wrap an LSTM layer, which means that my expected rnn_direction_mode should be 1 (bidirectional). However, the error message shows that rnn_direction_mode is 0, which indicates that the model expects a one-way RNN at execution time.

First of all, the above code is derived from the "chapter11_part02_sequence-models.ipynb" code content in Deep Learning with Python Second Edition -- FrancoisChollet I tried to run the author's source code and found the same error; In addition, I make sure that the code that divides the data into training set and verification set is correct. I have trained keras model with unary syntax successfully. My expectation is that the model can be successfully trained

0

There are 0 best solutions below