I am far from an expert on robotics, but I am currently investigating the feasibility of automating a key task in our factory.

We have an application where we would need image recognition and depth sensing to specify a coordinate in 3d space, where an object exists. Think of this as something hanging of a small tree / bush.

Then we need to make to robot arm to move there and using a cutting end effector to cut of the string to the object hanging.

As far as I understand it seems like most industrial robots today need to be pre programmed. Are there any good solutions where you could basically say to the robot:


where the (x,y,z) coordinates comes from a depth camera setup with image recognition. I am sorry if the questions is too non specific, but I would like to know what the current state of the technology is.

  • $\begingroup$ By experience there exists some framework out there to do what you describe, but its a vast topic. More info on your hardware and available skill set would provide a better question and proper answer. $\endgroup$
    – N. Staub
    Oct 8 '17 at 17:07
  • $\begingroup$ Welcome to Robotics Lukas Arvidsson, but I'm afraid that Unbounded Design Questions are off-topic because there are many ways to solve any given design problem. We prefer practical, answerable questions based on actual problems that you face, so questions which ask for a list of approaches or a subjective recommendation on a method (for how to build something, how to accomplish something, what something is capable of, etc.) are off-topic. Please take a look at How to Ask & tour for more information on how stack exchange works. $\endgroup$
    – Mark Booth
    Oct 9 '17 at 14:15
  • $\begingroup$ In short though, yes, this is possible, and having worked on such systems I can tell you that in the real world it's never as cheap or easy as you first think. In reality it's never just x,y,z, it's almost always (x,y,z,θx,θy,θz) and these position/angles are never quite the same. Having said that, if you can prototype it with Lego Mindstorms, you stand a good chance of being able to translate that to an industrial robot later. $\endgroup$
    – Mark Booth
    Oct 9 '17 at 14:21

Yes, with some caveats, using machine vision to get the 3D pose of an object and then using an industrial robot with a cutting tool to go to a location is within the capability of current technology.

Talk to your local automation and machine vision vendors.

Perhaps do a little searching on fruit picking robotics as the task is probably similar.


Not the answer you're looking for? Browse other questions tagged or ask your own question.