The body as input

Hands, everyone has them. Can be very intuitive and natural.

Hands, everyone has them. Can be very intuitive and natural. I should think about what makes some hand gestures natural for different inputs.

Hand

  • The fingers in relation to each other, beside each other, how they move side to side as gesture input.
  • The knuckles bending in relation to one another (counting on your hands in China).
  • Likely best because humans love their hands.
  • Also expensive real estate for the body. Hands are used lots for this reason. (There is prob a relationship between how dexterous it is and how much we use it day to day.)

Face

  • Quite demanding real estate, it is used socially.
  • We can call voice a subset of facial surface.

See Voice is not the ideal interface.

Top down or bottom up

Top down: move order is find most natural movements for the UI, then work backwards for what device can capture that.

Bottom up: research current/theorized devices and what inputs they allow you to capture, build the UI from there.

See the natural hand study, the input vocabulary and gesture references.

Gestures as commands

Select the text and then gesture. I could select the text and then just run commands or hotkeys on them, but having that gesture controlled wheel could be great!

I do feel like having limited gestures, "running commands" is useful. (See Growing an argument.)

Riding a horse bareback! As gesture input.

See What is this? (pointing to select), Voice is not the ideal interface (rapid fire input without text), Typing with six gestures and Will we gesture at all?.

The hyperobject

The best object is no object.

Input devices

There is either input or output.

Typing with six gestures

How do we type efficiently in public with only six gestures?

Will we gesture at all?

All this work on gestures, and yet I don't think we will be gesturing that much