Tensorflow Object detection development in IOS app - nonmaxsuppressionV2

Question

I am trying to detect object using ssd_mobilenet_v1_coco model. My own trained model file .pb file is used for detection. After successful build , click run button and I got the below error.

 "Not found: Op type not registered 'NonMaxSuppressionV2' in binary running on IPhone. Make sure the Op and Kernel are registered in the binary running in this process. "

I can executed and launch ios app for already trained .pb model file in below link. please give a solution to fix the above issues and launch ios app. https://github.com/JieHe96/iOS_Tensorflow_ObjectDetection_Example

Answer 1

The problem is exactly what the error says - The operation NonMaxSuppressionV2 is being used by the model (.pb file) you're using but it is not registered with tensorflow library while getting compiled for iOS platform. This is because tensor flow restricts a lot of operations [especially the ones usually required only for training] on iOS/Android platforms so that the size of compiled libraries are less.

To rectify the above problem you can do the following - Update the file present ops_to_register.h file present here.

Add the "NonMaxSuppressionV2Op<CPUDevice>" (Don't forget to add coma if you're adding in middle of array ) to kNecessaryOpKernelClasses array. Like this -

constexpr const char* kNecessaryOpKernelClasses[] = { 
"BinaryOp< CPUDevice, functor::add<float>>",
"BinaryOp< CPUDevice, functor::add<int32>>",
"AddNOp< CPUDevice, float>",
"NonMaxSuppressionOp<CPUDevice>",
//Added NonMaxSuppressionV2Op
"NonMaxSuppressionV2Op<CPUDevice>",
...
//Other operations
...
};

And also isequal(op, "NonMaxSuppressionV2") to constexpr inline bool ShouldRegisterOp(const char op[])

Like this -

constexpr inline bool ShouldRegisterOp(const char op[]) {
  return false
     || isequal(op, "Add")
     || isequal(op, "NoOp")
     || isequal(op, "NonMaxSuppression")
     //Added NonMaxSuppressionV2
     || isequal(op, "NonMaxSuppressionV2")
     || isequal(op, "Pack")
     //other stuff
     ...
  ;

After you modify this file re-run everything from scratch as mentioned in the quick-start section of repo's readme.

If you are still losing some other operations. Repeating the same procedure for them too will work.

Hope that helped.