We turned EmbeddingGemma 2 into a speech-to-text model
Points and comments are a snapshot, not live.
A team repurposed Google's EmbeddingGemma 2 embedding model into a fast on-device speech-to-text system.
The project, Embedding Gemma Transcribe, uses EmbeddingGemma 2's audio encoder unchanged with a small trained reader that maps audio slices to ~229,000 English words, plus a punctuation model. It runs entirely on a Mac's Neural Engine, processing an hour of audio in 12.6 seconds on a base M3 MacBook Air. The word error rate is about 5%, versus 2-3% for leading models. The base model was never trained for speech-to-text; only the reader was trained, in under two hours on 8×H100 GPUs.
What commenters are saying
Commenters found the approach clever as a proof of concept, but noted the 5% word error rate limits professional use. One commenter clarified that the reader is effectively a CTC head on the audio encoder, outputting EmbeddingGemma's text embeddings instead of a separately learned vocabulary. Another reported Apple's on-device model being 5x faster than whisper.cpp's large-v3-turbo but with somewhat worse performance. There was interest in multilingual support, particularly Hindi and Korean, and questions about whether the app is open source (it is not; only the base model is Apache 2.0).