← Public datasets
PUBLIC DATASET · SEGMENTED BY WORTEL

HoloAssist

Egocentric video of people operating and assembling everyday devices, segmented into atomic actions.

2
RECORDINGS TO PREVIEW
140
SEGMENTS
166
HOURS IN THE SOURCE DATASET
350
INSTRUCTOR-PERFORMER PAIRS
RECORDINGS

Pick a recording to preview

Each one plays with its segments on a synced timeline. Step through action by action, loop any clip and read the labels.

ABOUT THE DATASET

What HoloAssist is

HoloAssist is a large egocentric dataset from Microsoft and academic collaborators. A performer wears a mixed-reality headset and completes a physical task, such as making coffee or assembling furniture, while an instructor watches their view and guides them by voice.

We took recordings from the public release and segmented them in ELAN into single actions, so each clip holds one step of the task with frame-accurate boundaries.

WHAT WE ADDED
  • Atomic-action segmentation: one clip per action, with boundaries set frame by frame
  • Annotated in ELAN 7.1 by our trained team
  • Delivered as ELAN .eaf files or CSV
Source and licence

HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World

Xin Wang, Taein Kwon, Mahdi Rad, Bowen Pan, Ishani Chakraborty, Sean Andrist, Dan Bohus, Ashley Feniello, Bugra Tekin, Felipe Vieira Frujeri, Neel Joshi and Marc Pollefeys. ICCV 2023. Paper

The HoloAssist videos are shared under the Community Data License Agreement - Permissive, Version 2.0. They are the work of the HoloAssist authors, not Wortel AI. Our segmentation annotations are our own work. Read the CDLA-Permissive-2.0 licence.

Need HoloAssist segmented further?

Ask for these annotations, more recordings, or the same work done to your taxonomy, with per-hand descriptions if you need them.

Request annotations