2
$\begingroup$

As part of a research project, I'm creating synthetic data, using the Unity engine. I have the final pose of a robotic arm (6 params: 3 for orientation, 3 for location/position), and also a .ply file for the mesh. Is there any way I can use this information directly to render an image in Unity?

Update:

I was thinking of the following approach (though I am not sure how well this will translate to Unity). I have the Camera transform, so I can have the coordinates of the end effector in the Camera frame. My camera and light source are the same, so I was thinking that in a renderer (say Unity), I'd fix the origin of the 3D world as the camera+light point, and then if I could somehow input the end effector coordinates (x,y,z and thetax, thetay, thetaz) in the frame, that would align the arm in the camera (final position) as I need it, and then I'd render the image.

Also as of now, my ply file is really simple and has only 3 joints (end effector, of course, being the end point of the last). Is this approach viable in Unity?

.ply File, Image

|improve this question
$\endgroup$
  • $\begingroup$ you may get more people to read your post if you use a strategically placed typo ... accidentally add a y to arm in the title ;) $\endgroup$ – jsotola Jun 26 '19 at 6:24
  • $\begingroup$ So do you want to render the robot pose i.e. get an image of how the robot looks or do you want to render the image seen by the camera placed at the end of the robot? $\endgroup$ – 50k4 Jun 27 '19 at 8:44
  • $\begingroup$ Camera and light are fixed (in the 3d world origin). I want to move the robot arm in different orientations, locations, and get images of how that looks like from the fixed camera perspective $\endgroup$ – Ambareesh Jun 27 '19 at 21:39
1
$\begingroup$

You can convert the PLY files to OBJ fromat using MeshLab (free). You can import the OBJ files to Unity. More on how to do this can be found here.

I assume you have an obj file for all segments of the robotic arm, so you will have to recreate the chain of GameObjects that correspond to the linkages and attach the meshes to these. You have to take care of coordinate systems. You will have to have a Coordinate System (In Unity a Transfrom attached to a GameObject) at each joint (for motion) and probably, relative to the joint another one, relative to which the mesh vertex coordinates are expressed (they might coincide).

If you want to solve the inverse kinematics problem (calculate joint angles from end-effector pose) you can google for analytic solutions, iterative solutions or you can try the built-in Unity IK Solver.

If you want to render the image "as seen by a camera on the end-effector" you can just make the camera the child (or a component on) the last linkage of the robot.

|improve this answer
$\endgroup$
  • $\begingroup$ Thank you. I did the first step with Blender and converted the .ply to .flx for Unity. For the next part I was wondering if I could something simpler - I have updated my approach in the question. Do you think this might work? Or if Unity has a provision for this? $\endgroup$ – Ambareesh Jun 26 '19 at 22:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.