How to play Gesture Synth.

First sound / under one minute

Start with one open palm.

Open the Gesture Synth instrument, select Start Gesture Synth, and allow camera access. Stand or sit far enough back for your entire hand to fit inside the frame. When the status changes to RAISE A HAND, hold one palm toward the camera and spread your fingers naturally.

The hand skeleton should appear over the camera image and a note should sound. You do not need to wave quickly, point a finger, pinch, or choose a preset. A relaxed open hand is enough.

1. Start the instrumentSelect the main Start button. This click also unlocks browser audio.
2. Allow the cameraChoose Allow in the browser prompt. Gesture Synth never asks for microphone access.
3. Raise an open handFace the palm toward the lens and keep the wrist, fingertips, and space around the hand visible.
4. Move slowly at firstTravel sideways to hear the notes change. Close your hand when you want silence.

Control grammar / both hands

Four movements control the whole instrument.

Gesture Synth uses continuous position instead of a library of symbolic gestures. Both hands follow exactly the same rules, so the movement that works for your first note also works for every harmony.

X / horizontal

Choose a note

Move toward your left for lower notes and toward your right for higher notes. The mirrored camera view follows your movement.

Y / vertical

Shape tone

Raise the hand for a brighter, slightly stronger sound. Lower it for a warmer, softer sound.

Palm / open

Hold the gate

Keep the fingers comfortably extended to sustain the current note while you move through pitch and tone.

Palm / closed

Release the note

Make a loose fist to end the sound. Opening again begins a new note and creates rhythm.

The scale protects the melody

Horizontal space is divided into ten note zones drawn from D minor pentatonic. The scale leaves out the half-step clashes that often make random movement sound accidental. You can concentrate on timing and contour while the available notes remain compatible.

Musical movement / not random motion

Make a phrase, not a chase.

Fast movement is exciting, but a convincing phrase needs contrast. Begin with a clear rhythm: open for one count, close for one count. Then hold the hand open for longer notes and move through only two or three neighboring pitch zones.

Use horizontal motion for contour

A smooth path across the frame becomes an ascending or descending line. Pause over one region to establish a home note. Return to that area at the end of a phrase so the melody feels intentional.

Use height for expression

Vertical movement does not change the selected note. It opens or closes the low-pass filter and makes a small level adjustment. Lift a hand gradually to let a sustained note emerge; lower it to make the same note recede.

Use the fist as punctuation

Closing the hand is the equivalent of lifting a finger from a keyboard. Short open-close gestures create pulses. Longer open shapes create sustained lines. Leave real silence between phrases instead of keeping every note active.

Second voice / same language

Add harmony without changing the rules.

Bring a second open hand into the frame after the first hand feels predictable. Each tracked hand receives its own synth voice, pitch, tone, and stereo position. There is no lead-hand or chord-hand assignment.

1. Anchor one handHold the first hand near a comfortable low or middle note.
2. Enter with the secondOpen the second hand somewhere else in the frame to form an interval.
3. Move only oneKeep the anchor steady while the other hand explores nearby notes.
4. Trade the motionLet the moving voice settle, then move the anchor. This keeps the harmony readable.

If the voices feel crowded, separate your hands horizontally and avoid crossing them. The tracker preserves voice identity by following palm position, but clear spacing always produces the most stable result.

Five-minute practice / no theory required

Build control in three passes.

Pass 1: rhythm

Keep your hand in one pitch zone. Alternate open and closed shapes at a steady pace. Listen for clean beginnings and endings before adding movement.

Pass 2: melody

Hold the hand open and travel across three adjacent notes. Move slowly enough to hear each zone. End on the same note you started with.

Pass 3: expression

Repeat the three-note shape while changing height. Start low and warm, rise through the middle of the phrase, then settle back down. Add a second hand only after one hand feels deliberate.

Tracking conditions / clean signal

Give the camera an easy view.

Face a light source rather than placing it behind you. Keep your hand about an arm's length from the camera, with the palm broadly facing the lens. Plain backgrounds help, but consistent light and a fully visible hand matter more.

On a phone, place the device on a stable surface when possible. On a laptop, raise the screen until the camera sees your hands without forcing your shoulders upward. If you still cannot get a note, use the camera, sound, and tracking checklist.