Publications

You are viewing in chronological order, alternatively, you can check out my Google Scholar.

“SAM2Act: Integrating Visual Foundation Model with A Memory Architecture for Robotic Manipulation”

[Arxiv 2025]

Authors: Haoquan Fang, Markus Grotz, Wilbert Pumacay, Yi Ru Wang, Dieter Fox*, Ranjay Krishna*, Jiafei Duan*

[Project page][Paper]

“AHA: A Vision-Language-Model for Detecting and Reasoning over Failures in Robotic Manipulation”

[ICLR 2025]

Authors: Jiafei Duan, Wilbert Pumacay, Nishanth Kumar, Yi Ru Wang, Shulin Tian, Wentao Yuan, Ranjay Krishna, Dieter Fox, Ajay Mandlekar*, Yijie Guo*

[Project page][Paper]

“SAT: Spatial Aptitude Training for Multimodal Language Models”

[Arxiv 2024]

Authors: Arijit Ray, Jiafei Duan, Reuben Tan, Dina Bashkirova, Ross Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A. Plummer, Ranjay Krishna*, Kuo-Hao Zeng*, Kate Saenko*

[Project page][Paper]

“Manipulate-Anything:
Automating Real-World Robots using Vision-Language Models”

[CoRL 2024]

Authors: Jiafei Duan*, Wentao Yuan*, Wilbert Pumacay, Yi Ru Wang, Kiana Ehsani, Dieter Fox, Ranjay Krishna

[Project page][Paper][Code]

“RoboPoint: A Vision-Language Model for
Spatial Affordance Prediction for Robotic”

[CoRL 2024]

Authors: Wentao Yuan, Jiafei Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna Adithyavairavan Murali,Arsalan Mousavian, Dieter Fox

[Project page][Paper][Demo][Checkpoint][Code]

“EVE: Enabling Anyone to Train Robot using Augmented Reality”

[UIST 2024]

(24% Acceptance)

Authors: Jun Wang, Chun-Cheng Chang*, Jiafei Duan*, Dieter Fox, Ranjay Krishna

[Paper][Project]

“Octopi: Object Property Reasoning with Large Tactile-Language Models”

[RSS 2024, Oral Presentation (28%)]

Authors: Samson Yu, Kelvin Lin, Anxing Xiao, Jiafei Duan, Harold Soh

[Project Page][Code][Paper]

“THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation”

[RSS 2024, Oral Presentation (28%)]

Authors: Jiafei Duan*, Ishika Singh*, Wibert Pumacay*, Ranjay Krishna, Jesse Thomason, Dieter Fox

[Paper][Project Page][Code][Real-world setup]

“Selective Visual Representations Improve Convergence and Generalization for Embodied-AI”

[ICLR 2024, Spotlight (5%) ]

Authors: Ainaz Eftekhar*, Kuo-Hao Zeng*, Jiafei Duan, Ali Farhadi, Ani Kembhavi , Ranjay Krishna

[Paper][Project Page][Code]

“NEWTON: Are Language models Capable of Physical Reasoning”

[EMNLP Findings 2023]

Authors: Yi Ru Wang, Jiafei Duan, Dieter Fox, Siddhartha Srinivasa

[Paper][Project Page][Code][Dataset]

“AR2-D2:Training a Robot Without a Robot”

[CoRL 2023]

(Poster)

Authors: Jiafei Duan, Yi Ru Wang, Mohit Shridhar, Dieter Fox, Ranjay Krishna

[Paper][Project Page][My Talk][Dieter Fox’s Talk][Code]

“BOSS: A Benchmark for Human Belief Prediction in Object-context Scenarios”

ArXiv Preprint

Authors: Jiafei Duan*, Samson Yu*, Nicholas Tan, Li Yi, Cheston Tan

[Paper][Project Page][Code]

“Good Time to Ask: A Learning Framework for Asking for Help in Embodied Visual Navigation”

Ubiquitous Robots 2023 (Best Paper Award)

Authors: Jenny Zhang, Samson Yu, Jiafei Duan, Cheston Tan

[Paper][Project Page][Code]

“A Benchmark for Modeling Violation-of-Expectation in Physical Reasoning Across Event Categories”

[CogSci 2023]

(Poster)

Authors: Arijit Dasgupta, Jiafei Duan, Marcelo Ang, Yi Lin, Su-Hua Wang, Renée Baillargeon, Cheston Tan

[Paper] [Code][Dataset]

“ABCDE: An Agent-Based Cognitive Development Environment”

[CVPR 2022]

(Embodied AI Workshop)

Authors: Jieyi Ye, Jiafei Duan, Samson Yu, Bihan Wen, Cheston Tan

[Paper] [Demo]

“A Survey on Machine Learning Approaches for Modelling Intuitive Physics ”

[IJCAI-ECCAI 2022]

(Survey Track (Oral) , 18% Acceptance Rate)

Authors: Jiafei Duan*, Arijit Dasgupta*, Jason Fischer, Cheston Tan

[Paper] [Project Page] [Video]

“PIP: Physical Interaction Prediction via Mental Simulation with Span Selection”

[ECCV 2022]

(Poster Presentation, 28% Acceptance Rate)

Authors: Jiafei Duan*, Samson Yu*, Soujanya Poria, Bihan Wen, Cheston Tan

[Paper] [Project Page] [Code]

“AVoE: A Synthetic 3D Dataset on Understanding Violation of Expectation for Artificial Cognition ”

[NeurIPS 2021]

(Physical Reasoning and Inductive Bias Workshop)

Authors: Arijit Dasgupta, Jiafei Duan, Marcelo H.Ang Jr, Cheston Tan

[Paper] [Code] [Video]

“SPACE: A Simulator for Physical Interactions and Causal Learning in 3D Environments ”

[ICCV2021]

(Simulation Technology for Embodied AI Workshop)

Authors: Jiafei Duan, Samson Yu, Cheston Tan

[Paper] [Code] [Video] [Project Page]

“A Survey of Embodied AI: From Simulators to Research Tasks.”

[IEEE Transactions on Emerging Topics in Computational Intelligence]

(CIS Journal Featured Publication)

Authors: Jiafei Duan, Samson Yu, Tan Hui Li, Hongyuan Zhu, Cheston Tan

[Paper]

“ActioNet: An Interactive End-to-End Platform for Tasked-Based Data Collection and Augmentation in 3D Environment.”

[ICIP 2020]

(Poster Presentation, 47.9% Acceptance Rate)

Authors: Jiafei Duan, Samson Yu, Tan Hui Li, Cheston Tan

[Paper] [Code] [Video][Project Page]