The problem Augmented Reality Manipulator solves

• Industrial Robots and arm are quite expensive.
• Should be operated very carefully.
• Can potentially damage itself as well as the environment.
• Simulated Worlds made in simulators to test the arm require a lot of time to be built
• The Simulated worlds needs to be very precise wrt the real world to get a realistic feel

Using our augmented reality simulator people can just spawn any mechanical robot in augmented reality and test out operations, build applications, anything the user wants with just a push of a button

Manual Joint Control: This feature allows the user to select and manipulate individual joints of the arm in AR in an interactive way. The same movement is reflected in the actual/simulated arm in Gazebo.

Sync Joint angles: This feature can be used to sync the joint values in the AR environment with real world arm/simulated arm in gazebo. If the used changes the joint angles in the real arm manually, this fetch can be used to sync both arm the arms for further operations

Trajectory Planning: In this feature the user has access a ball which can be moved in the 3d space in AR. The used can move the ball to a desired position and press the fetch button. Moveit will plan a path for the arm to match the position of the end effector with the ball

Challenges we ran into

• UR5 Arm still spawns a little away from the image target due to lack of support in vuforia with articulation bodies.
• There is a 90 degree offset in the wrist 3 joint due to difference in xacro files used in real and AR
• The path planning cannot be used with real arm due to difference in xacro files
• An error regarding the difference in name for moveit msgs
• vurofia doesent have very good support for ROS_TCP_Connect
• Need of a pipeline to get the robot in unity