Segmentation Rules and Guidance

Six rules to remember

1. One clip = one action.
2. Split when the hand, the action or the object changes.
3. Keep one continuous action in one clip.
4. The hand must be visible at the start and end of every clip.
5. No clip is longer than 10 seconds.
6. Clips never overlap.

1. One clip = one action

A clip shows one complete action, from its start to its end.

Quick test: can you describe the clip in one short phrase, like "pick up the cup"? If you need two phrases, make two clips.

These are always separate clips: pick up, hold, carry, put down, open, close, turn on, turn off.

2. When to split

Start a new clip when the hand, the action or the object changes. Ask these four questions. If any answer is yes, start a new clip.

1. Did the hand change? Left to right, right to left, or one hand to both hands.

2. Did the action change?

3. Did the object change?

The same action on different items = 1 clip for each item, even if the items are the same type.

4. Do different steps repeat in a cycle?

More examples

3. When to keep one clip

Do not split one continuous action.

Same hand, same action, same object, no stop.

Two hands working together at the same time.

Small movements. Finger movements and grip changes belong to the main action.

4. Hand visibility

At the start and end of every clip, the working hand must be visible from wrist to fingers. For a two-hand action, one visible hand is enough.

During the clip, the hand should be visible at least 90% of the time. Do not make a clip when:

• The hand is hidden for 1 second or more without a break.
• The hand is hidden for more than 20% of the clip.
• You cannot tell which hand is working.
• The hand is behind glass or a mirror.
• It is too dark to see the hand.

Examples:

• Hand goes into a sink, cabinet, drawer or shelf for a short moment and the action clearly continues = keep the same clip.
• Hand is about to leave the frame = end the clip before it leaves.
• Hand keeps going in and out of view = clip only the visible parts. Do not join them into one clip.
• Hand disappears inside a drawer for 2 seconds = no clip for that part.

5. Clip length and timing

Maximum 10 seconds. Split long actions at least every 10 seconds, at a natural pause if there is one.

No overlaps. A red line on the timeline means two clips overlap. Move the clip edge with A/S or D/F until the red line is gone.

Hand label. Choose the hand that does the action: left, right or both. Left marked as right, or one hand marked as both, is a fatal error.

Very fast actions. An action under 0.5 seconds, such as quickly pressing a switch or quickly turning off a faucet, can join the action before or after it.

Exact start and end. Start the clip where the action starts. End it where the action ends.

Camera shake. End the clip before the shake starts.

6. What to record

Actions to record

• Pick up or put down an object, including food and drink.
• Hold an object.
• Carry an object while walking.
• Open or close a door, cabinet or drawer.
• Turn a faucet or switch on or off.
• Take an object out of a cabinet, shelf, tool bag or pouch, or put it in.
• Wash, cut, wipe, mop, sweep, rinse, wring, fold or tape an object.
• Touch an object on purpose.
• Apply soap to hands, wash hands with soap, dry hands with a towel.

Actions not to record

• Walking with empty hands.
• Empty or idle hands.
• Eating, drinking or smelling an item.
• Using a phone, laptop or touchscreen, or using a mouse or keyboard while sitting at a computer.
• Washing your own hands with water only, no soap or towel.
• Scratching.
• Taking things from clothing pockets, or putting them in.
• Putting on gloves, or adjusting gloves that are already on.
• Actions by another person.
• Smoking, or handling a gun.
• Hand movements behind glass or a mirror.
• Small finger movements that are part of a bigger action.

7. Task Error

Use Task Error only when the whole video cannot be annotated. If only one part is bad, annotate the good part.

Mark Task Error when the whole video has one of these problems:

• No hands, or almost no hands.
• Too dark, fully black or fully white.
• Ripples, flickering, or green, red or black bars and lines.
• Mostly phone use.
• Not a first-person view.
• The wearer has black gloves on.
• The camera wearer's face is visible. Only the nose visible is acceptable.

Faint black bars are acceptable and are not a Task Error.

8. Fatal errors

These four errors get the work rejected. An annotator who makes them three times is removed from the project.

1. More than one action in a clip. One clip holds the end of "turn on faucet" and the start of "pick up soap". It must be two clips.
2. Wrong hand label. Left hand marked as right, or one hand marked as both.
3. Hand not visible at the start or end. The first or last frame does not show the wrist and hand.
4. Overlapping clips. A clip starts before the previous clip ends.

Reviewers also mark these as serious errors: a clip over 10 seconds, over-splitting, a missed action, a clip for something on the "do not record" list, and a wrong Task Error.

9. Check before you submit

• One action in every clip.
• No action missed.
• Nothing from the "do not record" list.
• Hand visible at the start and end.
• No clip over 10 seconds.
• No overlaps.
• Hand labels correct.
• No over-splitting.
• Task Error only for a fully unusable video.

10. For reviewers

1. Check every task and every clip, not only the flagged ones.
2. Scrub the full timeline first to find missed actions.
3. For each clip, check: valid action, one action, start and end, hand label, visibility, length, overlap.
4. Write down each error: the task, the clip and the rule broken.
5. Look for mistakes that repeat across the account.
6. Decide. Approved: no mistakes. Rework: mistakes found; send the error list, and the annotator rechecks every task.
7. After rework, check the fixed clips first, then the rest of the account again.

11. Shortcuts