Error when i use onnx transformer on gpu for zero-shot-text classification

35 Views Asked by At

I am running a code for zero-shot-classification on google colab gpu using onnx version of transformer whose model name is "MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33" and am getting the error

RuntimeError: Error when binding input: There's no data transfer registered for copying tensors from Device:[DeviceType:1 MemoryType:0 DeviceId:0] to Device:[DeviceType:0 MemoryType:0 DeviceId:0]

I am attaching the code for reference. I have installed only the optimum[onnxruntime-gpu] and uninstalled onnxruntime. How can i resolve this?

Code:

import torch
from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline

model_gpu = "MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33"

class Zero_Shot_Classifier():
    
    def __init__(self) -> None:
        self.input = "Hello"
        self.candidate_labels = ['greeting','goodbye','anger']
        self.result = None
        self.classifier = None
        # self.input = input
        self.run_model()
        
    def run_model(self):
          
          tokenizer = AutoTokenizer.from_pretrained(model_gpu)
          o_model = ORTModelForSequenceClassification.from_pretrained(model_gpu,subfolder="onnx",file_name = "model.onnx",provider="CUDAExecutionProvider",use_io_binding=False)
          self.classifier = pipeline("zero-shot-classification",model=o_model,tokenizer = tokenizer,device = "cuda",batch_size = 4)
          self.result = self.classifier(self.input,self.candidate_labels,multi_label = True)
          torch.cuda.empty_cache()
          print(self.result)
            


answer  = Zero_Shot_Classifier()

Error:


RuntimeError: Error when binding input: There's no data transfer registered for copying tensors from Device:[DeviceType:1 MemoryType:0 DeviceId:0] to Device:[DeviceType:0 MemoryType:0 DeviceId:0]

I have installed only the optimum[onnxruntime-gpu] and uninstalled onnxruntime. Also, checked the versioning of cuda, pytorch and onnxruntime-gpu and made sure they are compatible.

0

There are 0 best solutions below