CoppeliaSim forums

Hello, I want to use a vision sensor to extract some coordinates from a object to move the end-effector of the robot to it. I need to detect semi circles in the image and I feel like the algorithms in VREP are not complex enough for this operation. If I'm correct, is there a way I export the image for the api and do the image processing outside of the simulator?

Thank you

You can write a plugin in C++ providing one or more custom Lua commands, and use whatever image processing algorithm (e.g. OpenCV, TensorFlow, ...) to extract the info you need.

Thanks! Is there any example of this kind that I can look for? I'm using remote API C++ already, can I just use opencv in the api?

Thank you again!

The Image plugin uses OpenCV 3 to perform simple operations and drawing on images, and already can read/write to/from vision sensors as well as capture devices (cameras).

I think it is a good starting point to extend with further OpenCV functions.

To implement new Lua commands remember to edit also the callbacks.xml file accordingly.