Pre-prototype
A hardware experiment: deliberate hand gestures and push-to-talk voice for controlling a Mac.
Why it exists
Mostly because building it is the fun part. The idea is a desk camera for where and a wristband for when: the camera follows your bare hand, and the band on the other wrist handles pinch, scroll, raise-to-talk and a small haptic tap to say it heard you.
Where it stands
Nothing has been soldered yet. What runs today is the software half: an Apple Watch stands in for the band, and raising your wrist to talk sends dictation to the Mac. It has been tested on the bench, not on a real day of work.
What it will not do
- Act on a gesture it is unsure of. Losing tracking cancels an action; it never commits it.
- Ask you to hold your arms up. Gestures stay low and small.
- Listen when you are not talking to it. The microphone is live only while you hold to talk.
- Need the internet for the core controls. The AI layer is optional.
White paper
- Software
- Swift. A macOS menu bar app, a watchOS app and a shared package
- Band
- Planned: Seeed XIAO nRF52840 Sense on Zephyr. Motion sensor, clutch button, haptics
- Camera
- Planned: a desk-mounted hand tracker
- Stage
- Research and pre-prototype
A learning project first. It is not a product, and it is not for sale.