Human Demostration
QuasiSim
We explore the dexterous manipulation transfer problem by designing simulators. The task wishes to transfer human manipulations to dexterous robot hand simulations and is inherently difficult due to its intricate, highly-constrained, and discontinuous dynamics and the need to control a dexterous hand with a DoF to accurately replicate human manipulations. Previous approaches that optimize in high-fidelity black-box simulators or a modified one with relaxed constraints only demonstrate limited capabilities or are restricted by insufficient simulation fidelity. We introduce parameterized quasi-physical simulators and a physics curriculum to overcome these limitations. The key ideas are 1) balancing between fidelity and optimizability of the simulation via a curriculum of parameterized simulators, and 2) solving the problem in each of the simulators from the curriculum, with properties ranging from high task optimizability to high fidelity. We successfully enable a dexterous hand to track complex and diverse manipulations in high-fidelity simulated environments, boosting the success rate by 11%+ from the best-performed baseline.
Manipulating daily objects (mouse).
Functional tool-using with non-trival object movements.
Functional tool-using.
Manipulations with non-trivial and subtle object movements.
Manipulating the thin bowl.
Functional interactions with rich and changing contacts.
Manipulations with subtle object movements (slight shacking).
Complicated tool-using with non-trivial object movements.
Manipulating the thin bowl.
Manipulating daily objects (bunny).
Manipulating daily objects (bunny).
Manipulating daily objects (mouse).
By optimizing DGrasp-Tracking through the physics curriculum, we can noticably enhance its performance,
lifting it from instances of failure to near completion in tracking tasks.
Human Demostration
DGrasp-Tracking
DGrasp-Tracking
w/ Curriculum
Human Demostration
DGrasp-Tracking
DGrasp-Tracking
w/ Curriculum
We conduct various ablation studies to validate the effectiveness of crucial designs in our method.
Human Demostration
Ours w/o
Analytical Sim.
Ours w/o
Local Force NN
Ours
We transfer the optimized trajectories to real dexterous Allegro hand manipulation sequneces.
Rotating a mouse.
Playing with a stapler.
Rotating a bottle.
Using a hammer.
Manipulating a flashlight.
Rotating a airplane.
Please contact us at xymeow7@gmail.com if you have any question.
@article{liu2024quasisim,
title={QuasiSim: Parameterized Quasi-Physical Simulators for Dexterous Manipulations Transfer},
author={Liu, Xueyi and Lyu, Kangbo and Zhang, Jieqiong and Du, Tao and Yi, Li},
journal={arXiv preprint arXiv:2404.07988},
year={2024}
}