Simultaneous Learning of Contact and Continuous Dynamics

November 2023   Bianchini

Project Documents

We published our work on data efficient model building through contact in a paper at the Conference on Robot Learning (CoRL) 2023 in Atlanta, GA. My co-authors are recent Penn PhD graduate Mathew Halm and our advisor, Professor Michael Posa. Our project simultaneously built contact and continuous dynamics of novel, possibly multi-link objects by observing their dynamics through collision-rife trajectory data.

We've made all of the below publicly available for any interested researchers/readers:

• Paper:  in Proceedings of Machine Learning Research.
• Code:  Github fork of dair_pll repository with code for training our Continuous + ContactNets (CCN) method and DiffSim/End-to-end alternatives.
• Dataset:  google drive link with over 500 tosses of a novel articulated object undergoing contact-rich trajectories.
• Project page:  google sites page made for anonymous review (this page contains all the same information).

Video 1:  My 1-minute spotlight talk, submitted and presented live at CoRL 2023.

Paper abstract

Robotic manipulation can greatly benefit from the data efficiency, robustness, and predictability of model-based methods if robots can quickly generate models of novel objects they encounter. This is especially difficult when effects like complex joint friction lack clear first-principles models and are usually ignored by physics simulators. Further, numerically-stiff contact dynamics can make common model-building approaches struggle. We propose a method to simultaneously learn contact and continuous dynamics of a novel, possibly multi-link object by observing its motion through contact-rich trajectories. We formulate a system identification process with a loss that infers unmeasured contact forces, penalizing their violation of physical constraints and laws of motion given current model parameters. Our loss is unlike prediction-based losses used in differentiable simulation. Using a new dataset of real articulated object trajectories and an existing cube toss dataset, our method outperforms differentiable simulation and end-to-end alternatives with more data efficiency.

Citation

@inproceedings{bianchini2023simultaneous,
  title = {Simultaneous Learning of Contact and Continuous Dynamics},
  author = {Bianchini, Bibit and Halm, Mathew and Posa, Michael},
  year = {2023},
  month = nov,
  booktitle = {Conference on Robot Learning (CoRL)},
  url = {https://openreview.net/forum?id=-3G6_D66Aua}
}

Dataset

Figure 1:  Automated data collection with Franka Panda arm.

We automated the collection of over 500 tosses of a two-link articulated object using a Franka Panda 7-DOF robotic arm.  We tracked the poses of the 2 links using TagSLAM then converted the two poses to minimal coordinates via an optimization problem.

Methods

Figure 2:  Our method for learning dynamics of an unknown object.

We aim to do model building, or system identification, of an unknown system with which a robot is interacting.  A Franka Panda arm repeatedly lifts and tosses an unknown articulated object onto a flat table, and cameras track the object's configuration over time.  This data is grouped into current and next state pairs.  We formulate an optimization problem that determines the best explaining contact forces given the observed dynamics transitions.  Our loss function penalizes those contact forces' violation of realistic contact dynamics, given the current belief of the physical parameters like geometry, friction, and inertia.  This loss informs the update of the physical parameters in a data efficient manner.  Then the converged physical parameters can be combined with a contact solver / simulator of choice to perform forward dynamics predictions.

Implicit representation and violation-based loss

Figure 3:  Explicit (left) models and implicit (right) models, during inference (top) and training (bottom).

In the figure above, the explicit and implicit representations during inference (simulation) are in blue in the top row at left and right, respectively.  Explicit models are typically trained with prediction-based losses (bottom left), as can implicit approaches as in DiffSim (bottom middle).  Our alternative trains implicit representations with a violation-based loss (bottom right).

Our approach utilizes both:

• Implicit representations:  leverages existing contact simulators parameterized by physical quantities of geometry, friction, and inertia to implicitly encode forward dynamics models.

• Violation-based loss:  uses the true observed dynamics transition to inform what contact forces must have occurred, not requiring forward simulation during training time.

This is in contrast to:

• Differentiable Simulation (DiffSim):  Uses implicit representations with a prediction-based loss.

• Explicit models (End-to-end):  Uses explicit representations with a prediction-based loss.

Geometry reconstruction results

Figure 4:  Geometry reconstruction results.

Above:  Geometry learning results on an articulated object.  Ground truth is in red, and the learned geometry is in blue.  Geometry is parameterized by a set of 8 vertices per link.

Dynamics predictions

Figure 5:  Continuous + ContactNets (CCN) before (left) and after (right) training.

Figure 6:  DiffSim before (left) and after (right) training.

Figure 7:  End-to-end before (left) and after (right) training.

Quantitative results

Figure 8:  Quantitative results.

We present four scenarios above, in order from left to right columns:

1. Articulated object with real data.

2. Cube with real data.

3. Asymmetric object in simulation with "vortex" augmented dynamics.

4. Articulated object in simulation with "gravity" impaired dynamics.

The three rows are the following metrics, from top to bottom (see our paper for more details on these calculations):

1. Average position error over a toss trajectory rollout started from an initial condition.

2. Average angular error over a toss trajectory rollout.

3. Volumetric geometry error.

Discussion