Speech-to-text models listen to recordings or live audio and output words, often with punctuation, timestamps and labels for who is speaking. They power captions, meeting notes, voice typing and voice assistants. Modern systems handle many languages and accents well, but noise, jargon and overlapping speakers still cause errors.
How it works
Audio is first converted into a visual-like representation of sound, usually a spectrogram showing how much energy is at each frequency over time, sliced into short frames of around tens of milliseconds. A neural network, now commonly a transformer, reads these frames and predicts the most likely sequence of text tokens. Older systems used separate acoustic, pronunciation and language models stitched together; modern end-to-end models learn the whole mapping from sound to text in one network, trained on very large collections of transcribed speech. Many also add punctuation, capital letters and timestamps for each word. Speaker diarisation is a separate step that groups speech by voice to label 'Speaker 1' and 'Speaker 2'. Systems run either in batch, processing a finished file, or in streaming mode, returning words within a fraction of a second for live captions. Accuracy is measured by word error rate (WER): the share of words substituted, deleted or inserted compared with a human transcript. Clear studio audio can achieve low error rates, while noisy rooms, strong accents, crosstalk and specialist vocabulary push errors up. Custom vocabulary lists can help with names and jargon.
Why it matters
Transcription turns audio and video into searchable, editable, reusable text. Creators use it for captions, which improve accessibility and help viewers who watch without sound, and to repurpose videos into blog posts and clips. Professionals get meeting notes and action points without typing. Businesses analyse support calls at scale. Text-based video editors also rely on it, letting you cut a video by deleting words from its transcript.
Where you will see it
- Auto-generating captions and subtitles for YouTube videos, Reels and Shorts.
- Recording meetings and producing transcripts, summaries and action items.
- Editing a podcast or video by deleting words in its transcript.
- Voice typing notes, messages and documents on a phone or computer.
Limits and common misconceptions
- Accuracy drops with background noise, overlapping speakers, strong accents and specialist terms; always proofread before publishing.
- Names, numbers and brand terms are often misheard, which can change meaning in legal, medical or financial contexts.
- Recording people may require their consent under privacy laws; check rules before transcribing calls or meetings.
How it differs from related terms
- Speech to text vs Text to speech (TTS) Speech-to-text turns spoken audio into written words; text-to-speech does the reverse, turning written text into synthetic spoken audio.
- Speech to text vs Natural language processing (NLP) NLP is the broad field of computers handling language; speech-to-text is one part of it, focused on converting sound into text.
Tools that use it
Frequently asked questions
How accurate is AI transcription?
With clear audio and one speaker, modern speech-to-text is very accurate and often needs only light corrections. Accuracy falls with background noise, people talking over each other, heavy accents, poor microphones and unusual words. A good microphone and a quiet room improve results more than switching tools.
What is word error rate?
Word error rate (WER) is the standard measure of transcription accuracy. It counts the words a system got wrong, missed or added, compared with a correct human transcript, divided by the total number of words. Lower is better. It is useful for comparing tools, but results depend heavily on the audio tested.
Can speech-to-text tell different speakers apart?
Many tools can, using a feature called speaker diarisation. It groups speech by voice and labels each section, such as Speaker 1 and Speaker 2, which you can then rename. It works best when people take turns and struggles when they talk over each other.
Related terms
- Reasoning model A language model trained to generate an internal chain of reasoning before giving its final answer, improving results on multi-step problems.
- RLHF Reinforcement learning from human feedback: a training method where people rank AI responses, and the model is tuned to produce the preferred kind.
- Synthetic data Data created by algorithms or AI models, rather than collected from real events, used to train, test or fine-tune AI systems.
- Temperature A generation setting that controls how random an AI model's word choices are: low values give predictable output, high values give varied output.
- Text to image Generative AI that turns a written prompt into a new image by learning how words relate to visual features.