Issue with 'ValueError' when computing metrics in NER using transformers library (Tuple is empty)

Question

Description: I am encountering issues while trying to compute metrics for Named Entity Recognition (NER) using the Hugging Face transformers library. The specific errors are 'ValueError' and I've been struggling to resolve them for quite some time.

Code: I have a function compute_metrics that takes an EvalPrediction object as input. The function aims to process the predicted logits and true labels, but I'm facing issues with the structure of the EvalPrediction object.

Here is the relevant part of the code:

from transformers.trainer_utils import EvalPrediction


def compute_metrics(eval_preds):
    """
    Compute evaluation metrics for Named Entity Recognition (NER) tasks.

    Parameters:
    eval_preds (EvalPrediction): An object containing the predicted logits and the true labels.

    Returns:
    A dictionary containing precision, recall, F1 score, and accuracy.
    """
    if not isinstance(eval_preds, EvalPrediction):
        raise ValueError("Invalid eval_preds structure. Expected an EvalPrediction object.")

    predictions = eval_preds.predictions

    if isinstance(predictions, tuple):
        if len(predictions) > 0:
            pred_logits = predictions[0]
        else:
            raise ValueError("Tuple predictions is empty.")
    else:
        pred_logits = predictions

    # Ensure pred_logits has at least two dimensions
    if len(pred_logits.shape) == 1:
        pred_logits = np.expand_dims(pred_logits, axis=0)

    # Rest of your code...
    # Get predicted labels by argmax along the token dimension
    pred_labels = np.argmax(pred_logits, axis=2)
    # ... (rest of your code)
      # Filter out padding tokens where label is -100
    predictions = [
        [label_list[pred] for (pred, label) in zip(pred_label, true_label) if label != -100]
        for pred_label, true_label in zip(pred_labels, labels)
    ]

    # Filter out padding tokens in true labels
    true_labels = [
        [label_list[label] for label in true_label if label != -100]
        for true_label in labels
    ]

    # Compute metrics
    results = metric.compute(predictions=predictions, references=true_labels)

    return {
        "precision": results["overall_precision"],
        "recall": results["overall_recall"],
        "f1": results["overall_f1"],
        "accuracy": results["overall_accuracy"],
    }

trainer = Trainer(
    model,
    args,
   train_dataset=tokenized_datasets["train"],
   eval_dataset=tokenized_datasets["validation"],
   data_collator=data_collator,
   tokenizer=tokenizer,
   compute_metrics=compute_metrics
)

trainer.train()

Error Messages: The errors I'm encountering are as follows:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-103-3435b262f1ae> in <cell line: 1>()
----> 1 trainer.train()

5 frames
<ipython-input-101-8c3cb1696dcb> in compute_metrics(eval_preds)
     21             pred_logits = predictions[0]
     22         else:
---> 23             raise ValueError("Tuple predictions is empty.")
     24     else:
     25         pred_logits = predictions

ValueError: Tuple predictions is empty.

Objective: I'm seeking assistance to understand and resolve these errors. I suspect the issue might be related to the structure of the EvalPrediction object and how I'm accessing the predicted logits.

Additional Information:

I'm using the Hugging Face transformers library for NER. I have verified that the input data and labels are correctly formatted. My full code is here