Zhi Cao

I am a researcher at Stanford University, fortunately advised by Dr. Huang Huang, Prof. Jiajun Wu, and Prof. Fei-Fei Li. Previously, I studied Computer Science at the University of Michigan, where I was advised by Prof. Jason J. Corso and Prof. Alexander Rodríguez.

My research focuses on robot learning from human and video data, especially dexterous manipulation, and world models. I build models that connect perception, prediction, and action: multimodal force-action foundation models for human-to-robot dexterity, visual-track world models that bridge video prediction and robot control, and controllable generation of hands and dynamic 3D characters.

profile photo

News

  • 2026.11: FACT received the Best Paper Award (3rd Prize) at the CoRL 2026 BiDexTool Workshop.
  • 2026.11: TrAct accepted as an Oral at the CoRL 2026 TAG Workshop.
  • 2026.10: Released FACT, a multimodal force-action-track foundation model for human-to-robot dexterity.
  • 2026.02: BiMotion accepted to CVPR 2026 as a Highlight.
  • 2025.09: HANDI accepted to the ICCV 2025 HANDS workshop.
  • 2025.05: Joined the Stanford Vision and Learning Lab.
Selected Publications

* Equal contribution  ·  † Equal advising

FACT: A Multimodal Force-Action-Track Foundation Model for Human-to-Robot Dexterity
Zhi Cao, Yining Hong, Jiaolong Yang, Yejin Choi, Li Fei-Fei, Jiajun Wu, Huang Huang
CoRL 2026 Workshop on Bimanual Dexterous Tool Use (BiDexTool) · Best Paper Award, 3rd Prize
Website · Paper

FACT jointly models force, action, and visual tracks to transfer dexterous skills from human demonstrations to robot hands, enabling contact-rich tasks such as playing the harp.

TrAct: Bridging Robot Control and Visual Prediction with Visual Tracks
Zhi Cao*, Howard Ji*, Kevin Zhang*, Kuangzhi Ge, Li Fei-Fei, Jiajun Wu†, Huang Huang†
CoRL 2026 Workshop (TAG) · Oral · Website · Paper

TrAct uses visual tracks as a shared interface between robot control and video prediction: it proposes action-track pairs, predicts their visual outcomes, and selects actions with a vision-language reward model.

T²Mem: Learning Test-Time Memory for Robotics
Yize Liu, Huang Huang, Yining Hong, Zijian Du, Zhi Cao, Li Fei-Fei, Jiajun Wu
arXiv 2026 · Website · Paper

T²Mem equips robot policies with a memory that is learned and updated at test time, letting them solve tasks that depend on what happened earlier in an episode.

BiMotion: B-spline Motion for Text-guided Dynamic 3D Character Generation
Miaowei Wang, Qingxuan Yan, Zhi Cao, Yayuan Li, Oisin Mac Aodha, Jason J. Corso, Amir Vaxman
CVPR 2026 · Highlight · Website · Paper

BiMotion represents character motion with B-spline curves, producing smooth and expressive text-guided dynamic 3D characters.

HANDI: Hand-Centric Text-and-Image Conditioned Video Generation
Yayuan Li, Zhi Cao, Jason J. Corso
ICCV 2025 Workshop (HANDS) · Website · Paper

HANDI generates videos with detailed hand appearance and motion from an image and an action description, using an automatically inferred motion area and a hand refinement loss.

Other Manuscripts Under Review

  • ReAmp: Reliability Amplification for Embodied Agents through Test-Time Scaling and Self-Selection
    Yize Liu, Jacky Kwok, Yining Hong, Luis Pabon, Jadelynn Dao, Zhi Cao, Chongkai Gao, Yejin Choi, Leonidas Guibas, Azalia Mirhoseini, Marco Pavone
  • Test-Time Solver for Trustworthy 4D Novel View Synthesis
    Miaowei Wang, Zhi Cao, Jingwei Xu, Oisin Mac Aodha, Amir Vaxman
  • Counterfactual Time Series Diffusion Guided by Expert Models
    Wenhao Mu, Zhi Cao, Mehmed Kerem Uludag, Alexander Rodríguez