The idea
Map a calibrated brow raise, jaw motion, wink, smile, or head pose to app controls or facial MIDI, with haptic acknowledgement for each command.
How it works
- Track the face with supported TrueDepth and ARKit face tracking and record a neutral baseline.
- Calibrate a small set of deliberate expressions to controls or MIDI actions.
- Require an intentional activation pattern and acknowledge each command with a haptic cue.
AI’s role: Help calibrate expressions the person intentionally chooses; deterministic thresholds and a deliberate activation step fire mapped commands.
First demonstration
Map three deliberate expressions to simple commands, add an explicit activation gesture, and measure false triggers during ordinary conversation.
What to solve next
How can deliberate expressions remain distinct from involuntary facial movements across long sessions and different users?