I am running a code for zero-shot-classification on google colab gpu using onnx version of transformer whose model name is "MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33" and am getting the error
RuntimeError: Error when binding input: There's no data transfer registered for copying tensors from Device:[DeviceType:1 MemoryType:0 DeviceId:0] to Device:[DeviceType:0 MemoryType:0 DeviceId:0]
I am attaching the code for reference. I have installed only the optimum[onnxruntime-gpu] and uninstalled onnxruntime. How can i resolve this?
Code:
import torch
from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline
model_gpu = "MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33"
class Zero_Shot_Classifier():
def __init__(self) -> None:
self.input = "Hello"
self.candidate_labels = ['greeting','goodbye','anger']
self.result = None
self.classifier = None
# self.input = input
self.run_model()
def run_model(self):
tokenizer = AutoTokenizer.from_pretrained(model_gpu)
o_model = ORTModelForSequenceClassification.from_pretrained(model_gpu,subfolder="onnx",file_name = "model.onnx",provider="CUDAExecutionProvider",use_io_binding=False)
self.classifier = pipeline("zero-shot-classification",model=o_model,tokenizer = tokenizer,device = "cuda",batch_size = 4)
self.result = self.classifier(self.input,self.candidate_labels,multi_label = True)
torch.cuda.empty_cache()
print(self.result)
answer = Zero_Shot_Classifier()
Error:
RuntimeError: Error when binding input: There's no data transfer registered for copying tensors from Device:[DeviceType:1 MemoryType:0 DeviceId:0] to Device:[DeviceType:0 MemoryType:0 DeviceId:0]
I have installed only the optimum[onnxruntime-gpu] and uninstalled onnxruntime. Also, checked the versioning of cuda, pytorch and onnxruntime-gpu and made sure they are compatible.