Pybullet Quickstart Guide

PyBullet

Quickstart Guide

Erwin Coumans, Yunfei Bai, 2017/2018

Visit the forums.

Introduction 2

Hello PyBullet World 3

connect, disconnect 3

stepSimulation 13

setRealTimeSimulation 14

getBasePositionAndOrientation 14

resetBasePositionAndOrientation 15

Transforms: Position and Orientation 15

getAPIVersion 17

Controlling a robot 17

Base, Joints, Links 17

getNumJoints, getJointInfo 18

setJointMotorControl2/Array 19

getBodyUniqueId, removeBody 26

createConstraint, removeConstraint,

changeConstraint 26

getNumConstraints,

getConstraintUniqueId 28

getConstraintInfo/State 28

getDynamicsInfo/changeDynamics 29

setTimeStep 31

setPhysicsEngineParameter 31

resetSimulation 32

changeVisualShape, loadTexture 39

Collision Detection Queries 39

getOverlappingObjects, getAABB 40

getContactPoints, getClosestPoints 41

rayTest, rayTestBatch 42

getCollisionShapeData 43

Inverse Dynamics, Kinematics 44

calculateInverseDynamics 44

calculateJacobian, MassMatrix 44

calculateInverseKinematics 46

Reinforcement Learning Gym Envs 47

Environments and Data 47

Train and Enjoy: DQN, PPO, ES 50

Virtual Reality 53

getVREvents,setVRCameraState 53

getKeyboardEvents, getMouseEvents 59

Build and install PyBullet 60

Support, Tips, Citation 63

Introduction

PyBullet is an easy to use Python module for physics simulation for robotics, games, visual

effects and machine learning. With PyBullet you can load articulated bodies from URDF, SDF,

computation, forward and inverse kinematics, collision detection and ray intersection queries.

The Bullet Physics SDK includes PyBullet robotic examples such as a simulated Minitaur

quadruped, humanoids running using TensorFlow inference and KUKA arms grasping objects.

lines and text). PyBullet has cross-platform built-in client-server support for shared memory,

UDP and TCP networking. So you can run PyBullet on Linux connecting to a Windows VR

server.

PyBullet wraps the new Bullet C-API, which is designed to be independent from the underlying

physics engine and render engine, so we can easily migrate to newer versions of Bullet, or use

a different physics engine or render engine. By default, PyBullet uses the Bullet 2.x API on the

CPU. We will expose Bullet 3.x running on GPU using OpenCL as well. There is also a C++ API

similar to PyBullet, see b3RobotSimulatorClientAPI.

Hello PyBullet World

Here is a PyBullet introduction script that we discuss step by step:

connect, disconnect

After importing the PyBullet module, the first thing to do is 'connecting' to the physics simulation.

PyBullet is designed around a client-server driven API, with a client sending commands and a

physics server returning the status. PyBullet has some built-in physics servers: DIRECT and

GUI. Both GUI and DIRECT connections will execute the physics simulation and rendering in

the same process as PyBullet.

Note that in DIRECT mode you cannot access the OpenGL and VR hardware features, as

mode does allow rendering of images using the built-in software renderer through the

getConnectionInfo

Given a physicsClientId will return the list [isConnected, connectionMethod]

connect using DIRECT, GUI

The DIRECT connection sends the commands directly to the physics engine, without using any

transport layer and no graphics visualization window, and directly returns the status after

other PyBullet command.

The commands and status messages are sent between PyBullet client and the GUI physics

simulation server using an ordinary memory buffer.

It is also possible to connect to a physics server in a different process on the same machine or

on a remote machine using SHARED_MEMORY, UDP or TCP networking. See the section

about Shared Memory, UDP and TCP for details.

Unlike almost all other methods, this method doesn't parse keyword arguments, due to

backward compatibility.

For example:

App_SharedMemoryPhysics, App_SharedMemoryPhysics_GUI and the Bullet Example

connection. This will let you execute the physics simulation and rendering in a separate

library. This can be useful when using SSH tunneling from a machine behind a firewall to a

Linux, while running the physics server on Windows in Virtual Reality using HTC Vive or Rift.

loadURDF, loadSDF, loadMJCF

Universal Robot Description File (URDF). The URDF file is used by the ROS project (Robot

caching using setPhysicsEngineParameter(enableFileCaching=0).

The loadURDF arguments are:

loadSDF, loadMJCF

You can also load objects from other file formats, such as .bullet, .sdf and .mjcf. Those file

formats support multiple objects, so the return value is a list of object unique ids. The SDF

format is explained in detail at http://sdformat.org. The loadSDF command only extracts some

essential parts of the SDF related to the robot models and geometry, and ignores many

elements related to cameras, lights and so on. The loadMJCF command performs basic import

of MuJoCo MJCF xml files, used in OpenAI Gym. See also the Important note under loadURDF

saveWorld

You can create an approximate snapshot of the current world as a PyBullet Python file, stored

angles, object positions and environment for example in VR. Later you can just load the

PyBullet Python file to re-create the world. The python snapshot contains loadURDF commands

together with initialization of joint angles and object transforms. Note that not all settings are

stored in the world file.

The input arguments are:

The saveState command only takes an optional clientServerId as input and returns the state id.

The saveBullet command will save the state to a .bullet file on disk.

The restoreState command input arguments are:

Either the filename or state id needs to be valid. Note that restoreState will reset the positions

and joint angles of objects to the saved state, as well as restoring contact point information. You

need to make sure the objects and constraints are setup before calling restoreState. See the

16

getEulerFromQuaternion

The getEulerFromQuaternion input arguments are:

getEulerFromQuaternion returns alist of 3 floating point values, a vec3.

quaternion and output a list of 9 floats, representing the matrix.

PyBullet provides a few helper functions to multiply and inverse transforms. This can be helpful

17

getAPIVersion

You can query for the API version in a year-month-0-day format. You can only connect between

physics client/server of the same API version, with the same number of bits (32-bit / 64bit).

[0..getNumJoints()] Since the base is not a regular 'link', we use the convention of -1 as its link

18

getNumJoints, getJointInfo

r2d2.urdf this should return 15.

getNumJoints input parameters:

getJointInfo

19

the stepSimulation the physics engine will simulate the motors to reach the given target value

targetVelocity = targetVel,

20

POSITION_CONTROL and VELOCITY_CONTROL, and as an external force for

21

setJointMotorControlArray takes the same parameters as setJointMotorControl2, except

22

getJointStates is the array version of getJointState. Instead of passing in a single jointIndex, you

You can enable or disable a joint force/torque sensor in each joint. Once enabled, if you perform

23

You can also query the Cartesian world position and orientation for the center of mass of each

link using getLinkState. It will also report the local inertial frame of the center of mass to the

24

The relationship between URDF link frame and the center of mass frame (both in world space)

Example scripts (could be out-of-date, check actual Bullet/examples/pybullet/examples folder.)

25

You get access to the linear and angular velocity of the base of a body using getBaseVelocity.

The input parameters are:

The input parameters are:

applyExternalForce/Torque

external forces are cleared to zero. If you are using 'setRealTimeSimulation(1),

applications).

32

getPhysicsEngineParameters

33

startStateLogging/stopStateLogging

State logging lets you log the state of the simulation, such as the state of one or more objects

after each simulation step (after each call to stepSimulation or automatically after each

of objects. There is also the option to log the common state of bodies such as base position and

orientation, joint positions (angles) and joint motor forces.

All log files generated using startStateLogging can be read using C++ or Python scripts. See

quadruped_playback.py and kuka_with_cube_playback.py for Python scripts reading the log

files. You can use bullet3/examples/Utils/RobotLoggingUtil.cpp/h to read the log files in C++.

For MP4 video recording you can use the logging option STATE_LOGGING_VIDEO_MP4. We

plan to implement various other types of logging, including logging the state of VR controllers.

As a special case, we implemented the logging of the Minitaur robot. The log file from PyBullet

Bullet/examples/pybullet/examples/logMinitaur.py for an example.

34

The command will return a non-negative int loggingUniqueId, that can be used with

utilities to find out, or check out the C++ source code of the logging or Python dumpLog.py

35

PyBullet has both a build-in OpenGL GPU visualizer and a build-in CPU renderer based on

Output is the 4x4 view matrix, stored as a list of 16 floats.

36

computeProjectionMatrix

The input parameters are:

getCameraImage

The getCameraImage API will return a RGB image, a depth buffer and a segmentation mask

37

38

bridge your own rendering method with PyBullet simulation, and synchronize the world

transforms manually after each simulation step.

The input parameters are:

39

The physics simulation uses center of mass as a reference for the Cartesian world transforms,

localInertialFrame using the getLinkState API.

changeVisualShape, loadTexture

40

not recompute any contact point information.

getOverlappingObjects, getAABB

structures have some heuristic that enlarges the AABBs a bit (extra margin and extruded along

optionally a link index. (when you don't pass the link index, or use -1, you get the AABB of the

41

getContactPoints, getClosestPoints

The getContactPoints API returns the contact points computed during the most recent call to

stepSimulation. Its input parameters are as follows:

getClosestPoints

48

Reinforcement Learning Gym Envs

A suite of RL Gym Environments are installed during "pip install pybullet". This includes PyBullet

versions of the OpenAI Gym environments such as ant, hopper, humanoid and walker. There

are also environments that apply in simulation as well as on real robots, such as the Ghost

The source code of pybullet, pybullet_envs, pybullet_data and the examples are here:

https://github.com/bulletphysics/bullet3/tree/master/examples/pybullet/gym.

python -m pybullet_envs.examples.enjoy_TF_HumanoidBulletEnv_v0_2017may

python -m pybullet_envs.examples.kukaGymEnvTest

Environments and Data

49

env = e.RacecarGymEnv(isDiscrete=False ,renders=True)

50

51

It is also possible to access the data, such as URDF/SDF robot assets, Wavefront .OBJ files

from the pybullet_data package. Here is an example how to do this:

Train and Enjoy: DQN, PPO, ES

For discrete Gym environments such as the KukaBulletEnv-v0 and RacecarBulletEnv-v0 you

can use OpenAI Baselines DQN to train the model using a discrete action space. Some

examples are provided how to train and enjoy those discrete environments:

OpenAI Baselines will save a .PKL file at specified intervals when the model improves. This

.PKL file is used in the enjoy scripts:

52

You can train various pybullet environments using TensorFlow Agents PPO. First install the

required Python packages: pip install gym, tensorflow, agents, pybullet, ruamel.yaml

python -m pybullet_envs.agents.train_ppo --config=pybullet_pendulum --logdir=pendulum

pybullet_pendulum

pybullet_doublependulum

pybullet_ant

pybullet_minitaur

tensorboard --logdir=pendulum --port=2222

Reality). If you start a local GUI physics server, the visualizer (bullet_client.py) will automatically connect

53

In a similar way you can train and visualize the Minitaur robot:

Here is an example video of the Minitaur gait: https://www.youtube.com/watch?v=tfqCHDoFHRQ

We will add some description how to get started with PyTorch and pybullet. In the meanwhile, see this

54

See also the vrBullet quickstart guide.

Oculus Rift Touch controller support. OpenVR is currently

working on Windows, Valve is also working on a Linux version.

See also https://www.youtube.com/watch?v=VMJyZtHQL50 for

an example video of the VR example, part of Bullet, that can be

customize this small script to point to the location of Python etc. Make sure to switch to

App_PhysicsServer_SharedMemory_VR*.exe. By default, this VR application will present an

The output parameters are:

55

Bullet/examples/pybullet/examples/vrTracker.py to track HMD and generic tracker.

setVRCameraState

VR Camera track an object, such as a vehicle.

56

PyBullet has some functionality to make it easier to debug, visualize and tune the simulation.

This feature is only useful if there is some 3D visualization window, such as GUI mode or when

connected to a separate physics server (such as Example Browser in 'Physics Server' mode or

standalone Physics Server with OpenGL GUI).

addUserDebugLine will return a non-negative unique id, that lets you remove the line using

removeUserDebugItem.

You can add some 3d text at a specific location using a color and size. The input arguments

are:

57

removeUserDebugItem. See also pybullet/examples/debugDrawItems.py

addUserDebugParameter lets you add custom sliders to tune parameters. It will return a unique

id. This lets you read the value of the parameter using readUserDebugParameter. The input

parameters of addUserDebugParameter are:

64

● By default PyBullet uses OpenGL 3. Some remote desktop environments and GPUs

Support, Tips, Citation

}

(and future Bullet 4.x API) behind this C-API. So if you use PyBullet or the C-API

At the moment this is not exposed, so you would need to either turn of gravity

acceleration for all objects, and manually apply gravity for the objects that need it.

65

Question: How to scale up/down objects?

Answer: You can use the globalScaleFactor value as optional argument to loadURDF and

loadSDF. Otherwise scaling of visual shapes and collision shapes is part of most

file formats, such as URDF and SDF. At the moment you cannot rescale objects.

5) Use a small amount of compliance for wheels using the <stiffness value="30000"/>

6) Use the double precision build of Bullet, this is good both for contact stability and

66

Question: What kind of constant or threshold inside Bullet, that makes high speeds impossible?