Is there a way to use a specific Pytorch model image processor in C++?

37 Views Asked by At

I would like to use a pretrained image segmentation AI model developed with Pytorch in my C++ program.

For that I have to convert it into TorchScript in order to be able to use it in my said program, which I did.

My problem is that before I can use the model, I must first process the images with the Image Processor of the said model, and I must also process them subsequently with this same image processor.

How could I use this processor without having to recode it in C++?

Is there a way to include the processing operations in the same TorchScript as the model?

Or to make another TorchScript that I could call before and after the model?

Here is how I use the model in my Python program (that I am looking to translate in C++) :

from transformers import Mask2FormerImageProcessor, Mask2FormerForUniversalSegmentation
 
image_processor = Mask2FormerImageProcessor.from_pretrained("facebook/mask2former-swin-large-mapillary-vistas-semantic", do_resize=False)
model = Mask2FormerForUniversalSegmentation.from_pretrained("facebook/mask2former-swin-large-mapillary-vistas-semantic")
 
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
model.to(device)
 
img = cv2.imread("path_img")
images = [img]
 
inputs = image_processor(images, return_tensors="pt")
 
with torch.no_grad():
    outputs = model(**inputs.to(device))
 
pred_maps = image_processor.post_process_semantic_segmentation(outputs, target_sizes=[(image.shape[0], image.shape[1]) for image in images])

I tried to convert the Image Processor class to a Torchscript by tracing but it didn't worked.

0

There are 0 best solutions below