DECA Entrepreneurship Innovation Plan · Demo v3.1.1

Sign in ASL. Read English. Hear it spoken.

SignSpeak is a feasibility demo: someone signs on camera. A short isolated sign, or a few signs with a pause after each, can use the dedicated 200-class BiLSTM. Gemini cleans those glosses into English, and the site speaks the text aloud. Songs and unclear clips use Gemini on the full video. It is a student proof of concept, not a certified interpreter.

Open the camera demo How it works

v3.1.1What's new in v3.1.1Patch notes

Longer webcam clips now stay small enough to reach Gemini video, and a few signs in a row are less likely to be thrown away for one soft piece. One-word clips are unchanged.

The camera records at 640×480 / 15 fps / 480 kbps so a 30-second clip stays under Vercel’s upload limit. If a recording is still too large, the app says so instead of failing later.
Gemini video retries once with the other container type (webm ↔ mp4) when the first mime guess is wrong. Unknown magic bytes no longer block the fallback.
Several-signs reading pads more generously, drops wrist twitches, and uses slightly looser gates (or soft top-1 guesses) so a real webcam phrase can still become English. Isolated-sign gates stay 55% / 0.15 / 0.75.
If translation still fails, the error names one tip — pause between signs, or sign one short word — and does not blame Google.

What judges will see

One loop, from webcam to spoken English.

1. Grant camera access and watch the live preview.

2. Tap Start signing, sign for a few seconds, then stop.

3. English text appears. The browser automatically speaks it.

No Gemini key? The demo still runs in labeled mock mode with a sample sentence, so the UX can be walked through without an account.

How the proof of concept works

The capture, translation, and voice steps are split on purpose so a DECA presentation can pause on each one.

1.Sign on camera

Allow the webcam, then record a short phrase in American Sign Language. The clip auto-stops after 30 seconds.

2.A dedicated model reads the sign

A short isolated sign goes to the 200-class ASL Citizen BiLSTM; several signs with a pause between them are read one by one. Gemini turns those glosses into English. Songs and unclear clips use Gemini on the full video.

3.The site speaks English

Your browser’s built-in speech engine reads the translation aloud. Replay it any time with one tap.

Honest limits

The dedicated model is an RL-fine-tuned 200-class ASL Citizen BiLSTM (~90.3% test top-1). It reads one isolated sign at a time. Pause between vocab signs and it can stitch a short phrase; fluent signing, songs, and conversation still use Gemini video. Gemini is also the fallback when confidence is low or landmarks fail. Accuracy varies with lighting, camera angle, and whether live MediaPipe matches the training keypoints. This proves the product loop is demoable. It is not a substitute for a human interpreter.