Robot Arm Pick-and-Place with PyBullet

Simulate a KUKA robot arm picking up objects using inverse kinematics.

Introduction

30 Second Summary

Every time a robot arm in a factory picks up a part and places it somewhere else, it solves a surprisingly complex math problem. It needs to figure out exactly how to bend each of its joints to reach the right spot in space.

In this project, you will build a 3D physics simulation where a KUKA robot arm picks up a red cube and places it on a target zone. You'll use Python and PyBullet to control the arm, compute joint angles with inverse kinematics, and watch the whole thing play out in a real-time 3D viewer.

What You'll Build

You'll run a Python script and watch a robot arm autonomously reach down to a pedestal, grab a red cube, carry it across the workspace, and set it down on a green target zone. All of this happens in a live 3D window on your machine.

By the end of this project, you'll have:

  • A live 3D physics simulation running on your machine with gravity, collisions, and motor control that you can spin around and inspect from any angle.
  • A robot arm you can command to reach any position in its workspace using inverse kinematics, hitting targets with sub-millimeter accuracy.
  • A complete pick-and-place routine where the arm approaches, grasps, lifts, transports, and releases a cube at a target location.
  • Secret Mission: an optional challenge to add obstacle collision detection to the workspace.

Are there any prerequisites?

You should be comfortable with intermediate Python concepts like functions, loops, and lists. No robotics or physics simulation experience is needed.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

Set Up PyBullet and Open Your First Physics World

Every robot needs a world to move in. Before you can load a robot arm or tell it where to reach, you need a physics simulation with gravity, a ground plane, and a 3D viewer to watch what happens.

In this step, you'll install PyBullet and write a short script that opens a live 3D physics window. This confirms your environment works before you add any robots.

In this step, get ready to:
  • Install PyBullet, the physics simulation library.
  • Create the project folder and open it in VS Code.
  • Write and run hello_pybullet.py to see your first 3D physics world.
Install PyBullet and create the project folder

You need a project folder on your Desktop to keep all the scripts you'll write, and PyBullet installed so Python can run physics simulations.

  • Press **Cmd+Space** (macOS) or the **Super** key (Linux/Windows) to open your search bar.
  • Type Terminal and press **Enter** to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir robot-arm-sim
cd robot-arm-sim
  • Confirm the folder was created by running:
ls ~/Desktop

You should see robot-arm-sim listed in the output.

  • Install PyBullet by running:
pip install pybullet==3.2.7

The install takes a minute or two. You'll see a progress bar followed by Successfully installed pybullet-3.2.7 at the end.

Install failing or taking a long time?

PyBullet compiles C++ code during installation, so it needs a C compiler. If you see errors about missing build tools, install them with sudo apt install build-essential and try the install again.

Make sure you're using Python 3.10 or higher. Check with python3 --version.

help me troubleshoot my PyBullet installation error

What is PyBullet?

PyBullet is an open-source physics engine for robotics simulation. It handles gravity, collisions, and motor control so you can test robot behavior without real hardware.

The pybullet_data module comes bundled with the install. It provides built-in URDF robot models, including the KUKA arm you'll use later.

  • Open VS Code in your project folder by running:
code .

VS Code opens with an empty robot-arm-sim folder in the Explorer sidebar.

Command not found?

If code is not recognized, open VS Code manually from your application menu, then click **File**, then **Open Folder**, and select the robot-arm-sim folder on your Desktop.

help me open my project folder in VS Code on Linux

Write and run the hello world script

Your first script will connect to PyBullet's 3D GUI, load a ground plane, set gravity, and run the simulation for two seconds. This proves your environment is fully working.

  • In VS Code, press **Cmd+N** (macOS) or **Ctrl+N** (Linux/Windows) to create a new file.
  • Save it immediately with **Cmd+S** (macOS) or **Ctrl+S** (Linux/Windows) and name it hello_pybullet.py.
  • Copy the following code into hello_pybullet.py:
# Import PyBullet and its built-in data module
import pybullet as p
import pybullet_data
import time

# Start a physics simulation with a 3D GUI window
physics_client = p.connect(p.GUI)
p.setAdditionalSearchPath(pybullet_data.getDataPath())
p.setGravity(0, 0, -9.81)

# Load the ground plane from a built-in URDF model
plane_id = p.loadURDF("plane.urdf")

print(f"PyBullet connected: client {physics_client}")
print(f"Ground plane loaded: ID {plane_id}")

# Run 480 simulation steps (2 seconds at 240 Hz)
for i in range(480):
    p.stepSimulation()
    time.sleep(1 / 240)

p.disconnect()

What does this code do?

  • p.connect(p.GUI) starts a physics simulation with a 3D viewer window. The GUI mode gives you a real-time visual display.
  • setAdditionalSearchPath tells PyBullet where to find its built-in models, like the ground plane and robot URDFs.
  • setGravity(0, 0, -9.81) sets Earth-like gravity pulling downward along the Z axis.
  • loadURDF("plane.urdf") loads a flat gray ground plane from PyBullet's built-in models.
  • The for loop runs 480 physics steps at 240 Hz, giving you about 2 seconds of real-time simulation before the script disconnects.
  • Save hello_pybullet.py with **Cmd+S** (macOS) or **Ctrl+S** (Linux/Windows).

Before you run this script, what do you think will happen? Will you see a terminal-only output, or something else?

  • Open the integrated terminal in VS Code by pressing **Ctrl+`** (backtick).
  • Run your script by entering this command:
python hello_pybullet.py

A 3D viewer window pops open showing a flat gray ground plane stretching to the horizon. In your terminal, you'll see two lines printed:

PyBullet connected: client 0
Ground plane loaded: ID 0

After about 2 seconds the window closes automatically as the script finishes. You just ran your first physics simulation.

No window appeared?

PyBullet's GUI mode requires a display server and OpenGL. If you see an error about OpenGL or display, make sure you're running on a desktop environment (not a headless server or SSH session).

If you see a ModuleNotFoundError, make sure you installed PyBullet in the same Python environment you're running the script with. Check with pip show pybullet.

help me debug why PyBullet's 3D window isn't opening

✔️ Awesome, I've got everything!

Great. Double check you've saved hello_pybullet.py and move on to the next step.

ⓧ I'd like to double check the full code

Here's the complete hello_pybullet.py for comparison:

import pybullet as p
import pybullet_data
import time

physics_client = p.connect(p.GUI)
p.setAdditionalSearchPath(pybullet_data.getDataPath())
p.setGravity(0, 0, -9.81)

plane_id = p.loadURDF("plane.urdf")

print(f"PyBullet connected: client {physics_client}")
print(f"Ground plane loaded: ID {plane_id}")

for i in range(480):
    p.stepSimulation()
    time.sleep(1 / 240)

p.disconnect()

Your physics world is alive. You've got gravity, a ground plane, and a 3D viewer. Next up, you'll load a real robot arm into this world and explore how its joints work.

Load the Robot Arm and Explore Its Joints

In the last step, you opened your first physics world and saw a flat ground plane. An empty world is not very exciting on its own.

Before you can command a robot arm to move, you need to understand how it is built. In this step, you'll load the KUKA IIWA, a 7-joint industrial robot arm that ships with PyBullet, and inspect how its joints are organized.

In this step, get ready to:
  • Load the KUKA IIWA robot arm into the simulation and see it in the 3D viewer.
  • Print a table of the robot's 7 joints showing their names, types, and limits.
  • Set up the camera for a comfortable viewing angle.
Load the KUKA IIWA robot and print joint information

Robot models in PyBullet are defined using URDF files. These files describe a robot's shape, joints, and physical properties in a standard format that any physics engine can read.

  • Create a new file called explore_robot.py in your project folder by running this command:
touch explore_robot.py
  • Confirm the file was created by running:
ls

You should see both hello_pybullet.py and explore_robot.py listed.

  • Open explore_robot.py in VS Code and paste the following code to load the robot into the simulation:
import pybullet as p
import pybullet_data
import time

# Connect to PyBullet with a 3D viewer window
physics_client = p.connect(p.GUI)
p.setAdditionalSearchPath(pybullet_data.getDataPath())
p.setGravity(0, 0, -9.81)

# Load the ground plane and the KUKA robot arm
p.loadURDF("plane.urdf")
robot_id = p.loadURDF("kuka_iiwa/model.urdf", [0, 0, 0], useFixedBase=True)

# Keep the simulation running until Ctrl+C
print("\nPress Ctrl+C to exit...")
try:
    while True:
        p.stepSimulation()
        time.sleep(1 / 240)
except KeyboardInterrupt:
    p.disconnect()

What does this code do?

  • p.loadURDF("kuka_iiwa/model.urdf", [0, 0, 0], useFixedBase=True) loads the KUKA IIWA robot model at the world origin. useFixedBase=True bolts the robot's base to the ground so it doesn't topple over.
  • The URDF file kuka_iiwa/model.urdf is a built-in model bundled with the pybullet_data module. No separate download needed.
  • The try/except KeyboardInterrupt block keeps the viewer open until you press Ctrl+C in the terminal.
  • Save explore_robot.py.
  • Run the script by running this command:
python explore_robot.py

A 3D viewer window opens with the KUKA robot arm standing upright on the ground plane. The camera angle might feel far away or awkward. You'll fix that shortly.

  • Press Ctrl+C in the terminal to close the simulation before making your next edit.

Don't see a 3D viewer window?

  • PyBullet's GUI mode requires OpenGL and a display server. Make sure you are running the script on your Linux desktop, not inside an SSH session.
  • Check that your graphics drivers are installed. NVIDIA or AMD proprietary drivers are recommended for hardware acceleration.
  • Help me debug why the PyBullet 3D viewer is not opening

What is a URDF file?

URDF stands for Unified Robot Description Format. It is an XML file that defines a robot's links (rigid body parts), joints (how parts connect and move), and physical properties like mass and inertia.

PyBullet includes several ready-made URDF models through the pybullet_data module, including the KUKA IIWA, Franka Panda, and even a small R2D2.

The robot is on screen, but you have no idea what is inside it. How many joints does it have? What are their names and movement ranges? Time to find out.

  • In explore_robot.py, find this line:
print("\nPress Ctrl+C to exit...")
  • Add the following joint inspection code directly above that line:
# Count the robot's joints
num_joints = p.getNumJoints(robot_id)
print(f"Robot loaded with {num_joints} joints:\n")

# Map joint type numbers to readable names
type_names = {0: "REVOLUTE", 1: "PRISMATIC", 2: "SPHERICAL", 3: "PLANAR", 4: "FIXED"}

# Print details for each joint
for i in range(num_joints):
    info = p.getJointInfo(robot_id, i)
    joint_name = info[1].decode("utf-8")
    joint_type = type_names.get(info[2], "UNKNOWN")
    link_name = info[12].decode("utf-8")
    lower_limit = info[8]
    upper_limit = info[9]
    print(f"  Joint {i}: {joint_name} ({joint_type})"
          f" | Link: {link_name}"
          f" | Limits: [{lower_limit:.2f}, {upper_limit:.2f}]")

What does this code do?

  • getNumJoints(robot_id) returns the total number of joints in the loaded robot model.
  • getJointInfo(robot_id, i) returns a tuple with all details for joint i. The code extracts the joint name (index 1), type (index 2), link name (index 12), and movement limits (indices 8 and 9).
  • type_names maps PyBullet's integer joint type codes to human-readable strings. The KUKA IIWA's 7 joints are all REVOLUTE, meaning they rotate around an axis.
  • Save explore_robot.py and run the script again:
python explore_robot.py

Your terminal now prints a table of 7 joints. Each row shows the joint's name, type, linked body part, and how far it can rotate in each direction.

  • Press Ctrl+C in the terminal to stop the simulation.

Not seeing the joint table?

  • Make sure the joint inspection code is placed above the print("\nPress Ctrl+C to exit...") line, not inside the while True loop.
  • Check that robot_id is defined before the joint loop. It should be assigned on the p.loadURDF("kuka_iiwa/model.urdf", ...) line.
  • Help me figure out why my joint information is not printing

What do joint limits tell you?

Each joint has a lower limit and upper limit measured in radians. These define how far the joint can rotate in each direction.

A joint with limits [-2.96, 2.96] can rotate about 170 degrees in either direction. The robot physically cannot move past these limits, just like a human elbow cannot bend backward.

Set up the camera for a good viewing angle

The default camera in PyBullet is usually too far away or aimed at an awkward angle. You can reposition it with a single function call.

  • In explore_robot.py, find this line:
robot_id = p.loadURDF("kuka_iiwa/model.urdf", [0, 0, 0], useFixedBase=True)
  • Add the following camera setup code directly below that line:
# Position the camera for a clear view of the robot
p.resetDebugVisualizerCamera(
    cameraDistance=1.5,
    cameraYaw=50,
    cameraPitch=-30,
    cameraTargetPosition=[0, 0, 0.4])

What does this code do?

  • resetDebugVisualizerCamera positions the 3D camera using four parameters: how far away it is, which direction it faces (yaw), its tilt angle (pitch), and the point it looks at.
  • cameraTargetPosition=[0, 0, 0.4] aims the camera slightly above ground level (0.4 meters up), which centers the robot's midsection in view.

Before you run the script, what do you expect to change about the 3D view compared to your last run?

  • Save explore_robot.py and run it:
python explore_robot.py

The camera is now angled at the robot from a comfortable distance. You can see the full arm standing on the ground plane, and the terminal still prints the table of 7 joints with their names, types, and limits.

  • Press Ctrl+C in the terminal when you are done looking.

Camera view looks the same as before?

  • Make sure the resetDebugVisualizerCamera call is placed after the robot_id = p.loadURDF(...) line and before the num_joints line.
  • Double check that the function name is resetDebugVisualizerCamera (no typos). Python will raise an AttributeError if the name is wrong.
  • Help me fix my PyBullet camera positioning

✔️ Awesome, I've got everything!

Great. Double check you have saved explore_robot.py.

ⓧ I'd like to double check the full code

Scroll up through your explore_robot.py file and confirm these sections are present in this order:

  • Imports, PyBullet connection, gravity, and loading the ground plane and robot.
  • The resetDebugVisualizerCamera call with distance, yaw, pitch, and target position.
  • The num_joints count, type_names dictionary, and the for loop that prints each joint's details.
  • The try/except KeyboardInterrupt simulation loop at the bottom.

You have a robot arm loaded, a table of its joints, and a clean camera view. Next up, you'll try to actually move the arm to reach a specific position in 3D space.

Move the Robot with Direct Joint Control

You loaded the KUKA robot and inspected its 7 joints. Now it is time to actually move it somewhere.

The most basic way to control a robot arm is to set each joint angle directly. There is a fundamental problem, though: you think in positions ("move to x=0.5, y=0.3, z=0.5") while joints think in angles ("rotate joint 3 by 1.0 radians"). In this step, you will try to reach a target by guessing angles, and discover just how impractical that is.

In this step, get ready to:
  • Set a target position and try to reach it by guessing joint angles.
  • Measure the distance between where the arm ends up and where you wanted it.
Set a target and try to reach it by guessing joint angles

You want the end of the robot arm (the end effector) to reach a specific point in 3D space. Your only controls are 7 joint angles. Can you guess the right combination?

  • Create a new file called joint_control.py in your project folder and add the following setup and target code:
import pybullet as p
import pybullet_data
import time
import math

physics_client = p.connect(p.GUI)
p.setAdditionalSearchPath(pybullet_data.getDataPath())
p.setGravity(0, 0, -9.81)
p.loadURDF("plane.urdf")
robot_id = p.loadURDF("kuka_iiwa/model.urdf", [0, 0, 0], useFixedBase=True)
p.resetDebugVisualizerCamera(
    cameraDistance=1.5,
    cameraYaw=50,
    cameraPitch=-30,
    cameraTargetPosition=[0, 0, 0.4])
end_effector_index = 6  # tip of the KUKA arm
target_pos = [0.5, 0.3, 0.5]  # where we want the arm to reach
# Draw a red crosshair at the target position
p.addUserDebugLine(
    [target_pos[0] - 0.05, target_pos[1], target_pos[2]],
    [target_pos[0] + 0.05, target_pos[1], target_pos[2]],
    [1, 0, 0], 3, 0)
p.addUserDebugLine(
    [target_pos[0], target_pos[1] - 0.05, target_pos[2]],
    [target_pos[0], target_pos[1] + 0.05, target_pos[2]],
    [1, 0, 0], 3, 0)
p.addUserDebugLine(
    [target_pos[0], target_pos[1], target_pos[2] - 0.05],
    [target_pos[0], target_pos[1], target_pos[2] + 0.05],
    [1, 0, 0], 3, 0)

What does this code do?

The first half is the same boilerplate from your previous scripts: connect to PyBullet, load the ground plane and KUKA arm, and position the camera.

  • end_effector_index = 6 identifies the tip of the arm (link 6, which you saw in your joint table in the previous step).
  • target_pos = [0.5, 0.3, 0.5] is the 3D coordinate you want the arm to reach (x, y, z in meters).
  • The three addUserDebugLine calls draw a red crosshair at the target so you can see exactly where you are aiming.
  • Below the crosshair code, add the motor control and simulation loop by pasting this code:
# Our best guess at joint angles to reach the target
manual_joint_angles = [0.4, 0.5, 0.0, -1.0, 0.0, 0.8, 0.0]

# Command each joint to move to its guessed angle
for i in range(7):
    p.setJointMotorControl2(
        bodyIndex=robot_id,
        jointIndex=i,
        controlMode=p.POSITION_CONTROL,
        targetPosition=manual_joint_angles[i],
        force=200)

# Let the arm settle into position
for step in range(500):
    p.stepSimulation()
    time.sleep(1 / 240)

What does this code do?

  • manual_joint_angles is a list of 7 numbers, one angle (in radians) for each of the KUKA's joints. These are guesses.
  • setJointMotorControl2 with POSITION_CONTROL tells each joint's motor to rotate to the guessed angle with a force of 200 N.
  • The simulation loop runs 500 physics steps, giving the arm about 2 seconds to settle into position.
  • Save joint_control.py and run it to confirm the arm moves by running:
python joint_control.py

You should see the 3D viewer open with the KUKA arm, a red crosshair marking the target, and the arm moving to a pose. The window closes after about 2 seconds because the script ends after the simulation loop.

Window closing too fast to see?

The window is only open for the duration of the 500-step loop (about 2 seconds). That is expected at this stage. In the next chunk, you will add code that keeps the window open so you can inspect the result.

If the window does not open at all, make sure your graphics drivers are up to date. PyBullet's GUI mode requires OpenGL.

Help me debug my PyBullet 3D viewer window

Measure and display the error

The arm moved to a pose, but did it actually reach the target? You need to read the arm's actual end-effector position and measure how far off it landed.

  • In joint_control.py, add the following measurement and exit code below the simulation loop:
# Read where the end effector actually ended up
link_state = p.getLinkState(robot_id, end_effector_index)
actual_pos = link_state[4]

# Calculate the Euclidean distance from target
distance = math.sqrt(
    sum((a - b) ** 2 for a, b in zip(actual_pos, target_pos)))

print(f"Target position:  [{target_pos[0]:.3f}, {target_pos[1]:.3f}, {target_pos[2]:.3f}]")
print(f"Actual position:  [{actual_pos[0]:.3f}, {actual_pos[1]:.3f}, {actual_pos[2]:.3f}]")
print(f"Distance error:   {distance:.3f} meters ({distance * 100:.1f} cm)")
print(f"\nGuessing joint angles is not practical for precise positioning!")

# Draw a blue line showing the gap between actual and target
p.addUserDebugLine(list(actual_pos), target_pos, [0, 0, 1], 2, 0)

print("Press Ctrl+C to exit...")
try:
    while True:
        p.stepSimulation()
        time.sleep(1 / 240)
except KeyboardInterrupt:
    p.disconnect()

What does this code do?

  • getLinkState reads the current position of link 6 (the end effector). Index [4] gives the world-frame position, which is the most accurate reading for where the tip actually is.
  • The Euclidean distance formula measures the straight-line gap between the actual position and the target.
  • A blue debug line is drawn from where the arm ended up to where you wanted it, making the miss visible in the 3D viewer.
  • The try/except KeyboardInterrupt loop keeps the window open so you can inspect the result. Press Ctrl+C in the terminal to close it cleanly.

Before you run this, think about it: do you expect the arm to land exactly on the red crosshair, or miss it? How close do you think 7 guessed angle values can get?

  • Save joint_control.py and run it again:
python joint_control.py

The arm moves to a pose, but the terminal reveals the truth: it missed the target by several centimeters. You should see output like this:

Target position:  [0.500, 0.300, 0.500]
Actual position:  [0.XXX, 0.XXX, 0.XXX]
Distance error:   0.XXX meters (X.X cm)

Guessing joint angles is not practical for precise positioning!

In the 3D viewer, look for the blue debug line stretching from the arm's tip to the red crosshair. That gap is the error. Guessing 7 joint angles to land on an exact 3D point is like trying to parallel park by independently turning the steering wheel, pressing the gas, and shifting gears without looking.

Why is this problem so hard?

This is the joint-space vs task-space problem. You think in task space ("move to position x, y, z"). The robot's motors operate in joint space ("rotate each joint by some angle").

With 7 joints, each affecting the arm's final position in a non-linear way, manually guessing the right combination is impractical. You need a mathematical function that converts a desired position into the exact joint angles. That function is called inverse kinematics.

  • Press Ctrl+C in your terminal to close the 3D viewer window.

Not seeing a blue debug line?

The blue line stretches from the arm's actual tip position to the red crosshair. If the arm's tip is blocking the view, try rotating the camera in the 3D viewer by clicking and dragging with your mouse.

If you see an error about math not being defined, check that import math is at the top of your file.

Help me debug my joint control script

✔️ Awesome, I've got everything!

Make sure you have saved joint_control.py.

ⓧ I'd like to double check the full code

Your joint_control.py file is too long to show in one block, so scroll up and confirm each section is present in this order:

  • Imports (pybullet, pybullet_data, time, math), PyBullet connection, ground plane, robot, and camera setup.
  • end_effector_index = 6 and target_pos = [0.5, 0.3, 0.5] followed by three addUserDebugLine calls drawing the red crosshair.
  • manual_joint_angles list, a for loop calling setJointMotorControl2, and a 500-step simulation loop.
  • getLinkState measurement, distance calculation, four print statements, a blue debug line, and the try/except KeyboardInterrupt exit loop.

The gap between guessed angles and precise positioning is exactly the problem that robotics had to solve. In the next step, you will use inverse kinematics to compute the exact joint angles needed to reach any target with sub-millimeter accuracy.

Reach Targets Precisely with Inverse Kinematics

In the last step, you tried to reach a target position by guessing joint angles. The arm ended up centimeters away from where you wanted it. That gap between thinking in positions and commanding in angles is one of the core problems in robotics.

Inverse kinematics solves this problem directly. You give it the position you want the end effector to reach, and it computes the exact joint angles needed to get there. In this step, you will use calculateInverseKinematics to nail the same target from Step 3, then push it further by tracing a moving path in real time.

In this step, get ready to:
  • Replace guessed joint angles with inverse kinematics to reach the target with sub-millimeter accuracy.
  • Trace a circle in 3D space to demonstrate real-time inverse kinematics.
Replace guessed angles with inverse kinematics

Instead of manually picking joint angles and hoping the arm lands close to the target, you will ask PyBullet to compute the angles for you. The calculateInverseKinematics function takes a target position and returns the joint angles that place the end effector there.

  • Create a new file called ik_control.py in your project folder and paste this setup code:
import pybullet as p
import pybullet_data
import time
import math

# Connect to PyBullet and set up the physics world
physics_client = p.connect(p.GUI)
p.setAdditionalSearchPath(pybullet_data.getDataPath())
p.setGravity(0, 0, -9.81)

# Load ground plane and robot arm
p.loadURDF("plane.urdf")
robot_id = p.loadURDF("kuka_iiwa/model.urdf", [0, 0, 0], useFixedBase=True)

p.resetDebugVisualizerCamera(
    cameraDistance=1.5,
    cameraYaw=50,
    cameraPitch=-30,
    cameraTargetPosition=[0, 0, 0.4])

# End effector is link 6, robot has 7 joints
end_effector_index = 6
num_joints = p.getNumJoints(robot_id)

# Same target position from Step 3
target_pos = [0.5, 0.3, 0.5]

What's different from joint_control.py?

This setup is nearly identical to the previous step's file. The key differences are that num_joints is now captured using getNumJoints (instead of hardcoding 7), and the manual_joint_angles list is gone.

You are using the exact same target position [0.5, 0.3, 0.5] so you can directly compare the accuracy.

  • Save ik_control.py.
  • Confirm the file was created by running:
ls

You should see ik_control.py alongside your other scripts.

  • Add the red crosshair marker and the inverse kinematics call below your target_pos line in ik_control.py:
# Draw red crosshair at the target position
p.addUserDebugLine(
    [target_pos[0] - 0.05, target_pos[1], target_pos[2]],
    [target_pos[0] + 0.05, target_pos[1], target_pos[2]],
    [1, 0, 0], 3, 0)
p.addUserDebugLine(
    [target_pos[0], target_pos[1] - 0.05, target_pos[2]],
    [target_pos[0], target_pos[1] + 0.05, target_pos[2]],
    [1, 0, 0], 3, 0)
p.addUserDebugLine(
    [target_pos[0], target_pos[1], target_pos[2] - 0.05],
    [target_pos[0], target_pos[1], target_pos[2] + 0.05],
    [1, 0, 0], 3, 0)

# Compute exact joint angles using inverse kinematics
joint_poses = p.calculateInverseKinematics(
    robot_id,
    end_effector_index,
    target_pos,
    maxNumIterations=100,
    residualThreshold=1e-5)

What does calculateInverseKinematics do?

You pass it the robot, the end-effector link index, and the position you want to reach. It returns a tuple of joint angles that will place the end effector at that position.

maxNumIterations=100 gives the solver up to 100 iterations to converge on a solution. residualThreshold=1e-5 sets the accuracy target. The default iteration count is only 20, which often leaves noticeable error.

  • Now add the code that applies those computed angles, lets the arm settle, and measures the result. Paste this below the calculateInverseKinematics call:
# Apply the IK-computed angles to each joint motor
for i in range(num_joints):
    p.setJointMotorControl2(
        bodyIndex=robot_id,
        jointIndex=i,
        controlMode=p.POSITION_CONTROL,
        targetPosition=joint_poses[i],
        force=200)

# Let the arm settle into position
for step in range(500):
    p.stepSimulation()
    time.sleep(1 / 240)

# Measure how close the end effector got to the target
link_state = p.getLinkState(robot_id, end_effector_index)
actual_pos = link_state[4]
distance = math.sqrt(
    sum((a - b) ** 2 for a, b in zip(actual_pos, target_pos)))

print(f"Target position:  [{target_pos[0]:.3f}, {target_pos[1]:.3f}, {target_pos[2]:.3f}]")
print(f"Actual position:  [{actual_pos[0]:.3f}, {actual_pos[1]:.3f}, {actual_pos[2]:.3f}]")
print(f"Distance error:   {distance:.4f} meters ({distance * 1000:.1f} mm)")

What does this code do?

  • The for i in range(num_joints) loop feeds the IK-computed angles into each of the 7 joint motors, just like the manual angles in Step 3.
  • The 500-step settling loop gives the physics simulation time to move the arm to the target.
  • getLinkState reads the actual end-effector position. Index [4] is the world-frame position of the link, which is the most accurate reading.
  • The Euclidean distance between actual and target positions is printed in both meters and millimeters.

Before you run this, think back to Step 3. Guessing joint angles left the arm several centimeters off target. What do you think the error will be this time with inverse kinematics computing the angles?

  • Save ik_control.py.
  • Run the script by running:
python ik_control.py

The 3D viewer opens, the arm moves to the target, and the terminal prints the accuracy. The distance error is under 1mm. Compare that to the centimeters of error from guessing joint angles in Step 3.

Window closes immediately?

The window opens only for the 500-step settling period (about 2 seconds) and then closes. That is expected at this stage. Focus on the terminal output for the accuracy numbers.

If you see no terminal output at all, check that your code pastes are in the right order. The calculateInverseKinematics call must come before the motor control loop.

Still stuck? Help me debug my ik_control.py script

Trace a circle to demonstrate real-time IK

Reaching a single static target proves IK works. The real power shows when you feed it a continuously moving target and the arm follows in real time.

  • Add this initialization code below the last print statement in ik_control.py:
print("\nTracing a circle with the end effector...")
t = 0
prev_pos = None
  • Now paste the circle tracing loop directly below those lines:
try:
    while True:
        t += 0.01
        # Compute a new target position on a circular path
        x = 0.4 + 0.2 * math.cos(t)
        y = 0.2 * math.sin(t)
        z = 0.5 + 0.1 * math.sin(t * 2)
        joint_poses = p.calculateInverseKinematics(
            robot_id,
            end_effector_index,
            [x, y, z],
            maxNumIterations=50)
        for i in range(num_joints):
            p.setJointMotorControl2(
                bodyIndex=robot_id,
                jointIndex=i,
                controlMode=p.POSITION_CONTROL,
                targetPosition=joint_poses[i],
                force=200)
        p.stepSimulation()
        # Draw a green trail behind the end effector
        link_state = p.getLinkState(robot_id, end_effector_index)
        current_pos = link_state[4]
        if prev_pos is not None:
            p.addUserDebugLine(
                list(prev_pos), list(current_pos),
                [0, 1, 0], 1, 5)
        prev_pos = current_pos
        time.sleep(1 / 240)
except KeyboardInterrupt:
    p.disconnect()

What does this loop do?

  • Each iteration increments t by 0.01, then uses cos(t) and sin(t) to compute a target that traces a circle in 3D space.
  • calculateInverseKinematics runs on every frame with maxNumIterations=50 (fewer iterations than the static target, because speed matters more than perfection when tracking a moving target).
  • addUserDebugLine draws a green line segment from the previous position to the current one, with a lifeTime of 5 seconds so older segments fade away.
  • The try/except KeyboardInterrupt block lets you press Ctrl+C to cleanly exit the simulation.
  • Save ik_control.py and run it:
python ik_control.py

The arm first reaches the static target (you will see the accuracy printed in the terminal). Then it starts tracing a smooth circle in the 3D viewer, leaving a green trail behind it.

  • Press Ctrl+C in the terminal when you are done watching the circle.

Arm not moving smoothly?

If the arm jerks or freezes, make sure the while True loop is inside the try: block. If the loop is outside the try/except, keyboard interrupts will crash the script without cleaning up.

If you see no green trail, check that prev_pos is initialized to None before the loop. The trail only draws after the first frame stores a position.

Need help? Help me debug my circle tracing loop

✔️ Awesome, I've got everything!

Great. Double check you have saved ik_control.py.

ⓧ I'd like to double check the full code

Scroll up and confirm that your ik_control.py contains all of these sections in order:

  • Imports, PyBullet connection, gravity, ground plane, and robot loading.
  • Camera setup and variable declarations (end_effector_index, num_joints, target_pos).
  • Three red crosshair debug lines at the target position.
  • calculateInverseKinematics call with maxNumIterations=100 and residualThreshold=1e-5.
  • Motor control loop applying IK angles, 500-step settling loop, accuracy measurement, and three print statements.
  • Circle tracing initialization (t = 0, prev_pos = None) followed by the try/except block with the while True loop.

Your robot arm can now reach any position you give it with sub-millimeter precision. Next up, you will use this power to build a complete pick-and-place task where the arm grabs an object and moves it to a target zone.

Build the Pick-and-Place Task

In the last step, inverse kinematics eliminated the guesswork of joint angles. Your KUKA arm can now reach any position in its workspace with sub-millimeter accuracy.

Now it is time to put IK to work on a real manipulation task. In this step, you will build a complete workspace with objects to interact with, then program the robot arm to autonomously pick up a red cube from a pedestal and place it on a target zone.

In this step, get ready to:
  • Build a workspace with a pedestal, a red cube, and a green target zone.
  • Write a reusable move_to helper function that wraps IK computation and motor control.
  • Program an autonomous pick-and-place sequence with grasping and release.
Build the workspace with a pedestal, cube, and target zone

Before the robot can pick anything up, it needs objects in its environment. You will create three workspace objects using PyBullet's shape API: a gray pedestal, a red cube sitting on top, and a green target zone on the ground.

  • Create a new file called pick_and_place.py in your project folder with the simulation setup and pedestal by copying the following code:
import pybullet as p
import pybullet_data
import time
import math

# Connect to PyBullet and load the simulation world
physics_client = p.connect(p.GUI)
p.setAdditionalSearchPath(pybullet_data.getDataPath())
p.setGravity(0, 0, -9.81)
p.loadURDF("plane.urdf")
robot_id = p.loadURDF("kuka_iiwa/model.urdf", [0, 0, 0], useFixedBase=True)

# Position camera for a wider workspace view
p.resetDebugVisualizerCamera(
    cameraDistance=1.8,
    cameraYaw=50,
    cameraPitch=-35,
    cameraTargetPosition=[0.3, 0, 0.3])
end_effector_index = 6
num_joints = p.getNumJoints(robot_id)

# Create a gray pedestal for the cube to sit on
pedestal_collision = p.createCollisionShape(
    p.GEOM_BOX, halfExtents=[0.1, 0.1, 0.15])
pedestal_visual = p.createVisualShape(
    p.GEOM_BOX, halfExtents=[0.1, 0.1, 0.15],
    rgbaColor=[0.6, 0.6, 0.6, 1])
p.createMultiBody(
    baseMass=0,
    baseCollisionShapeIndex=pedestal_collision,
    baseVisualShapeIndex=pedestal_visual,
    basePosition=[0.5, 0.15, 0.15])

What does this code do?

The first half is the same simulation setup you have used in every script so far. The camera is pulled back a bit further (cameraDistance=1.8) so you can see the full workspace.

The pedestal is the first object you build from scratch. createCollisionShape defines its physical shape for the physics engine. createVisualShape defines what it looks like. createMultiBody combines both into an actual object in the simulation. Setting baseMass=0 makes the pedestal static so it does not fall or move.

  • Save pick_and_place.py. The next chunk adds the remaining workspace objects so you can preview everything together.
  • Add the red cube, green target zone, and physics settling loop at the bottom of pick_and_place.py by copying the following code below the pedestal code you just added:
# Create a red cube on the pedestal (the object to pick up)
cube_size = 0.025
cube_visual = p.createVisualShape(
    p.GEOM_BOX, halfExtents=[cube_size] * 3,
    rgbaColor=[1, 0, 0, 1])
cube_collision = p.createCollisionShape(
    p.GEOM_BOX, halfExtents=[cube_size] * 3)
cube_id = p.createMultiBody(
    baseMass=0.1,
    baseCollisionShapeIndex=cube_collision,
    baseVisualShapeIndex=cube_visual,
    basePosition=[0.5, 0.15, 0.325])

# Create the green target zone where the cube will be placed
target_place = [0.5, -0.2, 0.025]
target_visual = p.createVisualShape(
    p.GEOM_BOX, halfExtents=[0.06, 0.06, 0.003],
    rgbaColor=[0, 1, 0, 0.6])
p.createMultiBody(
    baseMass=0,
    baseVisualShapeIndex=target_visual,
    basePosition=[target_place[0], target_place[1], 0.003])

# Let the physics simulation settle before moving
for _ in range(240):
    p.stepSimulation()
    time.sleep(1 / 240)

What does this code do?

The red cube uses the same collision-shape-plus-visual-shape pattern as the pedestal, but with baseMass=0.1 so it has weight and can be picked up. It is positioned at [0.5, 0.15, 0.325] which places it right on top of the pedestal.

The green target zone is a flat visual-only marker (halfExtents=[0.06, 0.06, 0.003]) with no collision shape, so the cube can land on top of it. The settling loop gives physics time to stabilize before the robot starts moving.

  • Save pick_and_place.py and preview the workspace by running:
python pick_and_place.py

The 3D viewer opens showing the robot arm standing next to a gray pedestal with a small red cube on top. A flat green rectangle sits on the ground nearby. The window closes after about a second because the script has no interaction loop yet.

Don't see the pedestal or cube?

  • Make sure the pedestal code and the cube code are both below the num_joints line. If they appear above the robot loading, the objects are created before the simulation world is ready.
  • Check that each createMultiBody call has both a baseCollisionShapeIndex and a baseVisualShapeIndex (except the target zone, which is visual only).
  • The window closes quickly. If you missed it, just run the script again and watch for the objects to appear.

Why isn't my PyBullet workspace showing the objects I created?

With the workspace ready, you need a reusable function that moves the end effector to any position. This wraps the IK computation, motor control, and simulation stepping you wrote separately in ik_control.py into a single call.

  • Add the move_to helper function at the bottom of pick_and_place.py, below the settling loop:
# Reusable helper: move the end effector to a target position
def move_to(target_pos, steps=400):
    joint_poses = p.calculateInverseKinematics(
        robot_id, end_effector_index, target_pos,
        maxNumIterations=100, residualThreshold=1e-5)
    for i in range(num_joints):
        p.setJointMotorControl2(
            bodyIndex=robot_id,
            jointIndex=i,
            controlMode=p.POSITION_CONTROL,
            targetPosition=joint_poses[i],
            force=200)
    for _ in range(steps):
        p.stepSimulation()
        time.sleep(1 / 240)

What does move_to do?

This function packages the three-step IK pattern into one reusable call. Given a target position, it computes the joint angles with calculateInverseKinematics, sends those angles to each motor with setJointMotorControl2, then steps the simulation 400 times (about 1.7 seconds) so the arm has time to reach the position.

  • Save pick_and_place.py. The next chunk calls move_to to start the pick-and-place sequence.
Program the pick-and-place sequence

The sequence breaks into seven steps: approach from above, lower to the cube, grasp it, lift, transport to the target zone, lower, and release. Each movement is a single move_to call. The grasp uses a fixed constraint that locks the cube to the end effector.

  • Add the approach and grasp sequence at the bottom of pick_and_place.py, below the move_to function:
# Read the cube's current position from the simulation
cube_pos, _ = p.getBasePositionAndOrientation(cube_id)

# Step 1: Move above the cube first
print("1. Moving above cube...")
move_to([cube_pos[0], cube_pos[1], cube_pos[2] + 0.15])

# Step 2: Lower to just above the cube
print("2. Lowering to cube...")
move_to([cube_pos[0], cube_pos[1], cube_pos[2] + 0.03])

# Step 3: Attach cube to end effector with a fixed constraint
print("3. Grasping cube...")
grasp_constraint = p.createConstraint(
    parentBodyUniqueId=robot_id,
    parentLinkIndex=end_effector_index,
    childBodyUniqueId=cube_id,
    childLinkIndex=-1,
    jointType=p.JOINT_FIXED,
    jointAxis=[0, 0, 0],
    parentFramePosition=[0, 0, 0.05],
    childFramePosition=[0, 0, 0])

# Let the grasp settle
for _ in range(120):
    p.stepSimulation()
    time.sleep(1 / 240)

Why use a fixed constraint for grasping?

Real robotic grippers use fingers, suction, or magnets. Simulating finger grasping with physics-based friction is finicky and often fails. createConstraint with JOINT_FIXED locks the cube to the end effector. Think of it as a magnetic gripper.

This keeps the focus on IK and manipulation sequencing rather than grasp mechanics. The parentFramePosition=[0, 0, 0.05] offset places the grasp point slightly below the end effector link so the cube attaches at a natural position.

  • Save pick_and_place.py and run the script to see the approach and grasp in action:
python pick_and_place.py

The 3D viewer opens and the arm moves above the red cube, lowers to it, and pauses for a moment after grasping. The window closes once the script finishes. In the terminal, you should see steps 1 through 3 printed.

Arm not reaching the cube?

  • Make sure the move_to function is defined above the line that calls it. Python reads top to bottom, so the function must exist before any move_to() call.
  • Check that the cube_pos line uses p.getBasePositionAndOrientation(cube_id) and not a hardcoded position. This reads the cube's actual position from the simulation.

Help me debug why my PyBullet robot arm is not reaching the cube position.

  • Add the transport, release, and accuracy reporting code at the bottom of pick_and_place.py, below the grasp settling loop:
# Complete the sequence: lift, transport, and release
print("4. Lifting cube...")
move_to([cube_pos[0], cube_pos[1], 0.7])
print("5. Moving to target zone...")
move_to([target_place[0], target_place[1], 0.7])
print("6. Lowering cube...")
move_to([target_place[0], target_place[1], target_place[2] + 0.03])
print("7. Releasing cube...")
p.removeConstraint(grasp_constraint)
for _ in range(240):
    p.stepSimulation()
    time.sleep(1 / 240)

# Measure placement accuracy and keep the window open
final_pos, _ = p.getBasePositionAndOrientation(cube_id)
distance = math.sqrt(
    (final_pos[0] - target_place[0]) ** 2 +
    (final_pos[1] - target_place[1]) ** 2)
print(f"\nCube placed at:  [{final_pos[0]:.3f}, {final_pos[1]:.3f}, {final_pos[2]:.3f}]")
print(f"Target was:      [{target_place[0]:.3f}, {target_place[1]:.3f}, {target_place[2]:.3f}]")
print(f"Placement error: {distance * 100:.1f} cm")
print("\nPick and place complete!")

print("Press Ctrl+C to exit...")
try:
    while True:
        p.stepSimulation()
        time.sleep(1 / 240)
except KeyboardInterrupt:
    p.disconnect()

What does this code do?

Steps 4 through 6 lift the cube to a safe height (z=0.7), transport it horizontally to the target zone, then lower it onto the target.

Step 7 calls removeConstraint to release the cube. After a settling period, the script reads the cube's final resting position with getBasePositionAndOrientation and calculates the Euclidean distance from the target to measure placement accuracy.

Before you run the complete script, think about what you expect to happen. Will the robot pick up the cube and place it accurately on the green target zone?

  • Save pick_and_place.py and run the full pick-and-place sequence:
python pick_and_place.py

Watch the robot arm approach the red cube on the pedestal, grasp it, lift it, carry it across the workspace to the green target zone, lower it, and release it. The terminal prints each step of the sequence and the final placement accuracy in centimeters.

Cube falling during transport?

  • Make sure the createConstraint call uses p.JOINT_FIXED as the jointType. Other joint types allow the cube to swing or slide loose.
  • Verify the grasp settling loop (for _ in range(120)) sits between the createConstraint call and the lift step. The constraint needs time to stabilize.
  • Check that removeConstraint only appears once, in step 7 after lowering to the target. Releasing too early drops the cube mid-air.

Why is the cube dropping during the pick-and-place sequence?

✔️ Awesome, I've got everything!

Great. Double check that you have saved pick_and_place.py.

ⓧ I'd like to double check the full code

The full pick_and_place.py file is built from the five code blocks above. Scroll up and confirm each section is present in this order:

  • Imports, PyBullet connection, robot loading, camera setup, and the gray pedestal creation.
  • Red cube creation, green target zone creation, and the settling loop.
  • The move_to helper function.
  • The approach and grasp sequence (steps 1 through 3 with createConstraint).
  • The transport, release, accuracy reporting, and the Ctrl+C exit loop at the end.

Secret mission

Detect Collisions with an Obstacle

Your robot arm moves through an empty workspace right now. Real robots share space with other objects, equipment, and people. Can you add an obstacle to the workspace and make the robot detect when it bumps into it?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally using free, open-source tools. There are no cloud resources, subscriptions, or ongoing costs.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Python scripts: hello_pybullet.py, explore_robot.py, joint_control.py, ik_control.py, pick_and_place.py, and collision_detection.py (local files in your project folder).
  • PyBullet Python package (pybullet, installed via pip).

Keep everything running

No action needed. Choose this if you plan to keep experimenting with your robot arm scripts or want to build on the simulation further.

  • Your Python scripts are local files on your machine. They cost nothing to keep.
  • The pybullet package stays installed and ready to use in any future Python project.
  • Run any of your scripts anytime with python followed by the script name.

Pause - I'll come back to this later

Everything in this project is local files and a pip-installed package. There are no running servers or background processes to shut down.

  • If you have any PyBullet 3D viewer windows open, close them or press Ctrl+C in the terminal to stop the running script.
  • Your scripts and the pybullet package will be right where you left them when you come back.

Delete - I don't want to use this again

Remove all project files and the PyBullet package to start fresh.

  • If a PyBullet script is still running, press Ctrl+C in the terminal to stop it first.
  • Navigate to the parent folder that contains your project directory by running:
cd ~/Desktop
  • Delete the entire project folder by running:
rm -rf pybullet-robot-arm

Check your folder name

If you named your project folder something different in Step 1, replace pybullet-robot-arm with whatever you called it. Run ls on your Desktop to find it.

  • Uninstall the PyBullet package by running:
pip uninstall pybullet
  • When prompted, type y and press Enter to confirm the uninstall.

Nice Work!

Nice Work!

You just built a fully autonomous robot arm simulation from scratch. A KUKA IIWA robot arm now picks up a red cube, carries it across the workspace, and places it on a green target zone, all on your local machine in real time.

You've learned how to:

  • Built a 3D physics simulation with gravity, collisions, and motor control using PyBullet, and loaded a real industrial robot arm from a URDF model.
  • Discovered the joint-space vs task-space problem firsthand by failing to reach a target with guessed joint angles, then solved it with inverse kinematics to achieve sub-millimeter accuracy.
  • Programmed a complete pick-and-place automation sequence: approach, grasp, lift, transport, place, and release, using constraint-based grasping and IK-driven movement.
  • Completed an optional Secret Mission to push your robotics simulation skills even further.

Ready to quiz yourself?