HandEdit: A Unified Benchmark for Egocentric Human-to-Robot Dexterous Hand Image Editing

Accepted by NeurIPS 2026

Zhenjie Yang^2,3*, Xingyu Jiao^1*, Guopeng Zhong³, Shuzhe Yang³, Shi Che³, Chao Wu¹, Chenyu Jiang¹,
Dongjie Zhang¹, Yideng Zhang³, Zheng Zhang³, Muyun Jiang⁵, Haisheng Su³, Shuang Jin², Donghang Zhang²,
Chao Yang⁶, Li Chen⁴, Hongyang Li⁴, Zuxuan Wu¹, Yu-Gang Jiang¹, Xiaosong Jia^1†, Junchi Yan^3†

¹ Fudan University ² Inspire Robots ³ Shanghai Jiao Tong University
⁴ The University of Hong Kong ⁵ Nanyang Technological University ⁶ Shanghai AI Laboratory
^* Core contribution ^† Corresponding authors
Contact: yangzhenjie@sjtu.edu.cn, jiaxiaosong@fudan.edu.cn

Paper Code Dataset Results BibTeX

Abstract

Robotic manipulation with dexterous hands is a cornerstone of Embodied AI, yet embodiment-aware teleoperation data remains costly to collect. Egocentric human videos offer a scalable alternative, but human and robotic hands differ profoundly in appearance, articulation, kinematics, and camera-relative geometry. HandEdit formulates this gap as an embodiment-aware image-editing problem. The benchmark asks an editor to replace the visible human hand or hand-arm region with a requested dexterous robot embodiment, while preserving object state, task semantics, contact relationships, viewpoint, and surrounding scene structure.

Embodiment Transfer

Apple cutting

HandEdit Dataset

EgoDex90M frames

338K clips · 500 household objects · 194 tasks

ARCTIC2.1M frames

Articulated objects · tool use · grasping

OakInk24.01M frames

75 objects · 150 tasks

HOI4D2.4M frames

Room-scale diversity · 54 tasks

HO-Cap656K frames

Pick-and-place · handover · tool use

Robot Embodiments

Benchmark Results

Model	Access	LPIPS ROI ↓	FID ROI ↓	Removal ↑	Struct ↑	ID ↑	Interaction ↑	VLM ↑

Radar plots comparing image editors on Hand-only and Hand-Arm metrics

Conclusion

GPT-Image-2 provides the strongest overall baseline.

VLM-based judgment is useful but not sufficient.

Perceptual quality alone does not guarantee editing task success.

The main challenge lies in embodiment-aware editing.

Qualitative Results

Qualitative comparison across commercial and open-source editors

BibTeX

@article{yang2026handedit,
  title={HandEdit: A Unified Benchmark for Egocentric Human-to-Robot Dexterous Hand Image Editing},
  author={Yang, Zhenjie and Jiao, Xingyu and Zhong, Guopeng and Yang, Shuzhe and Che, Shi and Wu, Chao and Jiang, Chenyu and Zhang, Dongjie and Zhang, Yideng and Zhang, Zheng and others},
  journal={arXiv preprint arXiv:2608.12122},
  year={2026}
}