My research focuses on robot learning from human and video data, especially dexterous manipulation,
and world models. I build models that connect perception, prediction, and action: multimodal
force-action foundation models for human-to-robot dexterity, visual-track world models that bridge
video prediction and robot control, and controllable generation of hands and dynamic 3D characters.
News
2026.11:FACT received the Best Paper Award (3rd Prize) at the CoRL 2026 BiDexTool Workshop.
2026.11:TrAct accepted as an Oral at the CoRL 2026 TAG Workshop.
2026.10: Released FACT, a multimodal force-action-track foundation model for human-to-robot dexterity.
2026.02:BiMotion accepted to CVPR 2026 as a Highlight.
2025.09:HANDI accepted to the ICCV 2025 HANDS workshop.
2025.05: Joined the Stanford Vision and Learning Lab.
Selected Publications
* Equal contribution · † Equal advising
FACT: A Multimodal Force-Action-Track Foundation Model for Human-to-Robot Dexterity Zhi Cao, Yining Hong, Jiaolong Yang, Yejin Choi, Li Fei-Fei, Jiajun Wu, Huang Huang CoRL 2026 Workshop on Bimanual Dexterous Tool Use (BiDexTool) · Best Paper Award, 3rd Prize Website · Paper
FACT jointly models force, action, and visual tracks to transfer dexterous skills from human demonstrations to robot hands, enabling contact-rich tasks such as playing the harp.
TrAct: Bridging Robot Control and Visual Prediction with Visual Tracks Zhi Cao*, Howard Ji*, Kevin Zhang*, Kuangzhi Ge, Li Fei-Fei, Jiajun Wu†, Huang Huang† CoRL 2026 Workshop (TAG) · Oral · Website · Paper
TrAct uses visual tracks as a shared interface between robot control and video prediction: it proposes action-track pairs, predicts their visual outcomes, and selects actions with a vision-language reward model.
T²Mem: Learning Test-Time Memory for Robotics
Yize Liu, Huang Huang, Yining Hong, Zijian Du, Zhi Cao, Li Fei-Fei, Jiajun Wu arXiv 2026 · Website · Paper
T²Mem equips robot policies with a memory that is learned and updated at test time, letting them solve tasks that depend on what happened earlier in an episode.
BiMotion: B-spline Motion for Text-guided Dynamic 3D Character Generation
Miaowei Wang, Qingxuan Yan, Zhi Cao, Yayuan Li, Oisin Mac Aodha, Jason J. Corso, Amir Vaxman CVPR 2026 · Highlight · Website · Paper
BiMotion represents character motion with B-spline curves, producing smooth and expressive text-guided dynamic 3D characters.
HANDI: Hand-Centric Text-and-Image Conditioned Video Generation
Yayuan Li, Zhi Cao, Jason J. Corso ICCV 2025 Workshop (HANDS) · Website · Paper
HANDI generates videos with detailed hand appearance and motion from an image and an action description, using an automatically inferred motion area and a hand refinement loss.
Other Manuscripts Under Review
ReAmp: Reliability Amplification for Embodied Agents through Test-Time Scaling and Self-Selection
Yize Liu, Jacky Kwok, Yining Hong, Luis Pabon, Jadelynn Dao, Zhi Cao, Chongkai Gao, Yejin Choi, Leonidas Guibas, Azalia Mirhoseini, Marco Pavone
Test-Time Solver for Trustworthy 4D Novel View Synthesis
Miaowei Wang, Zhi Cao, Jingwei Xu, Oisin Mac Aodha, Amir Vaxman
Counterfactual Time Series Diffusion Guided by Expert Models
Wenhao Mu, Zhi Cao, Mehmed Kerem Uludag, Alexander Rodríguez