grid search on parrot paraphraser

35 Views Asked by At

I am new to ML and programming for ML. I am trying to do a grid search on the parrot-paraphraser_for_t5 transformer from hugging face. There are two issues I am facing:

  1. I am not sure of the training_data format.
  2. I am not sure about how to mention the parameters for getting the best combination.

This is how I am currently specifying my trainer:

trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset = [
                {"input_text": "Paraphrase: 'The quick brown fox jumps over the lazy dog.'", "target_text": "'A speedy brown fox leaps over the lethargic canine.'"},
                {"input_text": "Paraphrase: 'I enjoy reading books.'", "target_text": "'Reading books is something I find pleasurable.'"},
                {"input_text": "Paraphrase: 'The weather is nice today.'", "target_text": "'Today's weather is pleasant.'"},
],
        data_collator = transformers.DataCollatorForSeq2Seq(tokenizer=tokenizer, padding=True)
    ) 

and below are my training arguments, placeholders for now.

training_args = TrainingArguments(
        output_dir="./results",
        per_device_train_batch_size=params["per_device_train_batch_size"],
        num_train_epochs=params["num_train_epochs"],
        learning_rate=params["learning_rate"],
    )

Huggingface website mentions that Parrot offers knobs to control Adequacy, Fluency and Diversity. Are these the hyper parameters I should be looking at? If so, how can I specify them in my code?

I am using T5Tokenizer and DataCollatorForSeq2Seq as my collator.

Please find the link to the hugging face transformer here and the link to my google colab code here.

Thanks in advance!

0

There are 0 best solutions below