Get your first sound
Show a number. Let it settle. Hear it play.
Open the instrument, select Start Gesture Synth, and allow camera access. When the status reads RAISE A HAND, face one palm toward the camera and extend one, two, three, four, or five fingers.
The number normally follows the visible count. The easiest path is index for 1, then add the middle, ring, and pinky for 2 through 4. A thumb on its own also counts as 1. Keep the thumb folded across the palm for 4; extend it beyond the index side only for 5. As an alternate 3, touch thumb and index into an OK loop while the other three fingers remain raised.
The center readout first says CONFIRMING. Keep the shape steady for a few video frames. When it changes to TRIGGERED, the large number and sound name show exactly what the synth received.
Start → allow camera → show 1–5 fingers → wait for TRIGGERED.
- Start the instrumentThe direct tap unlocks browser audio and then asks for camera access.
- Show one clear handKeep wrist, palm, and all fingertips visible. Separate extended fingers slightly.
- Read the confirmationA short progress bar rejects accidental shapes formed while changing counts.
- Rest without strainLower the hand to the bottom edge or hold a fist deliberately to release and re-arm.
Finger map
The same hand can play a melody or a progression.
In Melody, counts 1–5 play the first five notes of a major scale. In C, that is C, D, E, F, and G. You can change the key and octave in Sound Lab.
In Chords, the five shapes play I, vi, IV, V, and ii. In C, that becomes C, Am, F, G, and Dm. These functions cover the harmonic center, contrast, lift, tension, and a softer minor color without requiring chord theory.
- Finger count
- Pitch or chord
- Each stable count has a fixed musical meaning. Horizontal travel never sweeps through notes.
- Confirmation
- Prevents accidental notes
- The shape must remain steady briefly before it can trigger.
- Hand height
- Tone color
- Higher opens the filter and adds presence; lower sounds warmer.
- Bottom edge or fist
- Rest
- Release, relax the arm, and re-arm the same number for another attack.
Free Play
Start simple, then add color.
Choose Free Play for open performance. Begin in Melody with one hand and play a short contour such as 1–2–3–2–1. Let each count flash TRIGGERED before moving. Raise the hand slightly on the highest note, then lower it to rest.
Open Sound Lab to choose Airglass, Softpad, Pulsewire, Velvet, Bellveil, Reedline, or Stringloom. Output controls level; Brightness shapes the spectrum; Space adds reverb; Echo adds repeats; Motion adds chorus and stereo width; Visual Energy changes the size and number of stage effects.
When melody feels predictable, switch to Chords. A reliable first loop is 1–4–2–3, which becomes I–V–vi–IV in the selected key. Slow, even changes suit Softpad; shorter changes suit Pulsewire or Velvet.
Compare every voice and its best musical role in the Gesture Synth instruments guide.
Song Quest
Follow the next number like a rhythm game.
Choose Song Quest, select a melody, and choose Practice or Quest. Practice waits for each correct gesture and does not save a score. Ranked Quest waits until a hand is tracked steadily before it starts the countdown, then pauses if tracking is interrupted. A correct confirmed gesture plays the next melody note and earns PERFECT or GOOD, while a wrong or late gesture stays silent and breaks the combo.
SHOW marks the first number. CHANGE TO asks for a different confirmed number. HOLD means the next note uses the same number, so the shape may remain in place. A held repeat keeps the phrase moving as HOLD ASSIST, but only a fresh gesture attack adds ranked score, accuracy, and combo.
The current library contains complete single-line phrases of about 11 to 30 seconds. Each chart stores its own melody notes, timing, note lengths, and gesture cues. Gesture Synth does not bundle third-party recordings, MIDI files, modern arrangements, or lyrics.
Read the Song Quest guide, practice without a timer in the Practice walkthrough, or browse the playable song library.
Before playing, Listen to phrase previews the melody and Beat pulse keeps a quiet rhythmic reference. Each song also recommends a voice suited to its speed and note lengths. After a cleared Quest run, Share challenge creates a result card and challenge link in the browser. Anyone opening the link sees the challenge before camera or audio starts.
Keep the shape held to hear the repeated note, or release and re-form it when you want that beat to count toward a ranked combo.
Recording
Record the actual stage, effects, and synth audio.
Open the circular Record control, choose 9:16 for Shorts, Reels, and TikTok or 16:9 for landscape video, then choose 30, 60, or 120 seconds. After a three-count, Gesture Synth composes the camera, visual effects, live finger label, and synthesized audio into one local file. Song Quest recommends 60 seconds when a 30-second capture would cut off the count-in or final note.
Finished recordings remain available on the result screen and are also saved to My Works in this browser when storage permits. Select Share to open the system share sheet, or download the file when direct sharing is unavailable. Read the local recording and My Works guide for formats, storage limits, and privacy details.
Comfort and framing
Keep movement small and the camera useful.
- Support the deviceUse a stand or stable surface so framing does not move while you perform.
- Stay an arm's length awayThe tracker needs the full hand, not a close crop of fingertips.
- Use front lightSoft, even light improves finger separation and reduces recognition flicker.
- Return to restLower the arm between phrases. Do not keep it raised when no sound is needed.