Earlybird: predictive speech-to-text in 29M parameters

• Size29M · 16.8 MB29M parameters at 4 bits
• Word latency−90 msMedian, relative to the end of the word, on audiobooks. Other models: +90 to +400 ms
• Words predicted correctly before they end69%On audiobooks, counting only words not corrected later. Other models: 3–25%

Earlybird is a 16.8 MB on-device speech-to-text model that predicts each word before the speaker finishes saying it along with a confidence score. On audiobook speech, a typical word is predicted correctly 90 ms before it ends. Released streaming models predict it correctly 90–400 ms after.

Why earlier words matter

Streaming speech-to-text models output a word only after the speaker has finished it, typically 90–400 ms later. For a voice agent, that delay is time it could have spent starting a search, fetching a record, or planning a reply while the speaker is still talking.

Earlybird learns from audio cut off at random points, including partway through a word, and is asked to predict every word that is due by then in full, plus a guess of the next two words. Its raw token probabilities give a confidence for every word including the one still being spoken.

See it on real speech

Try out Earlybird and see how it compares to other models on real recordings. Each recording was fed to Earlybird and three released models in 80 ms chunks, as if it were live. At ¼× speed you can see Earlybird write each word while it is still being said. Darker blue means more confident.

Or try it yourself. This is the real 4-bit model, running on a small CPU server. Press the microphone and talk.

Results

I compare Earlybird with three released models on the same audio: Moonshine Tiny (34M parameters, 45 MB), Whistle (55M, 16.9 MB) and Whisper base (74M, about 145 MB). I streamed each recording into each model 80 ms at a time, as if it were live, and recorded the model’s text after every chunk. The times measure how much audio a model had to hear before it got the word right. Computing time depends on the hardware and is not included. A word’s word latency is the first moment the word is on screen, spelled correctly and in the right location, and stays unchanged after that. I measure it from when the speaker finishes the word, so a negative number means the model outputted the word before the speaker finished saying it.

On all six test sets, Earlybird predicts words correctly about 360 ms sooner than Moonshine Tiny, 490–580 ms sooner than Whistle and 160–200 ms sooner than Whisper base. It makes fewer errors than Moonshine Tiny on every set. Whistle is ahead on two sets, by at most 0.6 points, and Whisper base is the most accurate of the four. On the 8 English test sets of the Hugging Face Open ASR Leaderboard, Earlybird makes fewer errors than Moonshine Tiny on all 8 and roughly ties Whistle. Whisper base is more accurate, at about 9 times the size.

Earlybird is the only model under 1 GB that predicts the median word correctly before it ends. Its accuracy is in line with the other models under 100 MB: close to Whistle and Whisper tiny, and ahead of Moonshine Tiny.

How it’s trained

1. Start from Moonshine Tiny. I cut its vocabulary to the 19,007 tokens that English speech uses. That brings it to 29M parameters with almost no loss in accuracy.
2. Train on cut-off audio. Each training example is an utterance cut off at a random 80 ms step, which is what the model sees while streaming. The target is every word due by that point, an end marker, and a guess of the next two words. A word is due at whichever comes later: 300 ms before it ends, or 50 ms after it starts.
3. Learn from the teacher only where it’s right. Moonshine Tiny also acts as a teacher, but only on tokens where its top choice matches the label. Using it everywhere made the guesses less decisive.
4. Add padding frames. After the audio heard so far, the model sees three frames computed from silence, which lets it finish a word as if the speaker had stopped. This uses no future audio, and it moved the median from −50 to −90 ms.
5. Quantize to 4 bits. Quantization-aware training brings the weights to 4 bits, with the audio front end at 8 bits. The file is 16.8 MB, or 17.6 MB with the tokenizer and config. It runs about twice as fast as real time on one CPU thread.

Three training runs with different seeds agree to within 3 ms and 0.5 WER points on every test set.

Confidence: knowing when to trust an early word

Earlybird gives every word it shows a confidence, which measures the probability that the word is right. This is the model’s raw output, with no extra calibration step, and it matches how often the model is right: when Earlybird is at least 90% confident in a word it hasn’t heard the end of, it is right 91–96% of the time.

Related work

Zink et al. (2024) train an offline recognizer to predict masked words at the end of an utterance. Yusuf et al. (2024) run a language model, prompted with audio, ahead of a streaming recognizer. Chang et al. (2020) and Schwarz et al. (2023) show that early, stable partial transcripts help voice assistants. Earlybird makes the prediction inside a small streaming recognizer, and measures how well calibrated and how stable its words are, not only how early they come.

Next steps

• Close the accuracy gap. Whisper base, 2.5 times larger, still makes fewer errors on every test set except audiobooks, where they tie. The gap is widest with background chatter: at 10 dB, Earlybird gets 32% of words wrong and Whisper base 8.9%.
• Fewer rewrites. Showing words early means changing them more often: 1.3 changes per word on audiobooks (Whistle 0.2, Whisper base 0.4, Moonshine Tiny 2.1), and 2–6 on spontaneous or noisy speech. I want to bring that down without giving up earliness.
• Better next-word guesses. At 50% confidence or more, only about 1 word in 7 is guessed correctly before it ends.
• Voice commands. Commands to a voice agent are short and predictable, so they should be easier to guess early. Earlybird could be tuned on them specifically.

Links

• Model: Hugging Face
• Code: GitHub
• Paper: arXiv (link coming soon)
• Built on Moonshine Tiny by Moonshine AI (MIT)