Skip to content
Français

Speech AI: Transcribe, Understand and Speak

Turn phone calls into text and decisions, and build a phone line that answers by itself. Measure speech recognition honestly on phone-quality audio, compare voice detectors, recognisers, speaker splitting and voices on the same calls, build a small IVR that understands speech and keypad tones, and compare it with a voice agent.

Read every lesson as a guest, or sign in to enroll and track your progress.

Is this the right course for you?What this course covers and does not cover, and 4 courses to take for other needs.

This course teaches speech as a business system: audio as data, honest measurement of speech recognition, voice activity detection, local recognisers compared by slice, speaker splitting and timing, call reasons, summaries and compliance checks from transcripts, text-to-speech, a small IVR with speech and keypad tones that learners build and test, a voice agent with a latency budget, and privacy and misuse risks of voices. It does not teach how neural networks and transformers learn, prompt design from the start, or the platform that deploys and monitors models.

Take another course if you want to:

What you'll learn

By the end of this course, you will be able to:

  • Describe audio as data (sample rate, channels, bit depth, codecs, loudness, spectrograms), and prepare real recordings for speech models without losing information.
  • Measure speech recognition honestly: word and character error rate after a written normalisation, by slice, with the errors that matter to the business counted apart.
  • Build and compare voice activity detection, speech recognition, diarization and text-to-speech methods on the same audio, and choose one by error, speed, cost and privacy.
  • Turn transcripts into decisions (call reason, summary, compliance checks, keywords) and know when the transcript errors break the decision.
  • Build a small IVR that understands speech and keypad tones, confirms what it heard and hands over to a person, and test it with recorded calls.
  • Design a voice agent with a latency budget, streaming, turn-taking and interruption, compare it with the IVR on the same calls, and choose with evidence.
  • Run speech in production: batch and streaming, cost per audio hour, redaction of personal data in text and audio, consent and retention, and the risks of voice cloning and speaker verification.

About this course

Larkfield’s help desk takes about 4,000 phone calls a month in English and French. The calls are recorded with consent, but nobody can search them. Nobody knows why most people call, which promises the agents made, or which calls need a follow-up. At night, the phone simply rings out.

In this course, you follow Priya, Larkfield’s data scientist, Grace, the support lead, and Malik, the telephony engineer, as they turn calls into decisions. You start by measuring: you build a test set, write a normalisation guide, and count the errors that matter, such as a wrong order number, apart from the overall score. You test every method the same way, on the same calls, in one comparison table. In the command-spotting case study, you see a small, specialised model beat a large one where it counts.

Then you go further: voice detectors that find the speech and stop a recogniser from inventing text in silence, local recognisers of different sizes on a laptop CPU, speaker splitting, call reasons and summaries from transcripts, and compliance checks.

You build one project through the course: the Larkfield IVR, a small phone line that you call from a test phone in your browser. It understands speech and keypad tones, reads back the order number it heard, asks again after a miss and hands over to a person after two. You test it with recorded calls, then compare it with a voice agent and a hybrid on the same calls. You finish with the production work: cost per audio hour, card numbers removed from text and audio, consent and retention, and why a voice is not a password. Every Larkfield call is synthetic, made with open text-to-speech voices; no real customer call is used.

Before you start

You completed Machine Learning, From Problem to Reliable Model, or you have the same skills. You can read and change Python code, use notebooks and a terminal, and read JSON, and you know test sets, precision and recall, thresholds and slices. The deep learning and LLM API courses help but are not required. You need Python 3.12 on your computer, or a free Google or Kaggle account for the notebooks, and headphones. You need no key, no graphics card and no money.

Courses to take first: Machine Learning: From Problem to Reliable Model.

Who this is for

AI and ML engineers, backend developers and data scientists who must turn speech into text and decisions, such as call centres, meetings, voice notes and captions, or build a phone line or voice assistant, and technical leads who must choose between local and hosted speech models.

How we made this course

Experienced AI practitioners designed this course. AI helped us draft the content; people chose the teaching approach and the examples, and reviewed every lesson. The outputs that you see are the actual results of running the code. How we make our courses

Inside the course

Final module · choose at least oneCase studies

4 case studies

Four end-to-end case studies on real open speech. Recognisers for two languages, meeting transcripts with speakers, command spotting and an IVR for spoken bank requests. Complete at least one for the certificate; each one is a portfolio piece.

Complete at least one case study for the certificate. The others are optional. What each one covers

  1. Choose a recogniser for two languages120 min
  2. Meeting transcripts with speakers150 min
  3. Spot commands on a small device150 min
  4. An IVR for spoken intents150 min

After the case studiesFinal assignment

Use every module in one practice project. You collect evidence, and check it against the course outcomes.

  1. Final assignment180 min

Case studies

Realistic projects from start to finish, on real data, with every output explained. Each one makes a portfolio piece. Complete at least one for the certificate; the others are optional.

  1. Choose a recogniser for two languages2 hr

    Compare Vosk and Whisper sizes on real English and French speech from an open dataset by speaker group, and recommend one with error, speed and cost.

  2. Meeting transcripts with speakers2 hr 30 min

    Diarize and transcribe real recorded meetings from an open corpus, measure DER and WER, and make action-item notes.

  3. Spot commands on a small device2 hr 30 min

    Build a keyword spotter on an open spoken-commands set, compare a small CNN on spectrograms with Whisper tiny, and measure false alarms per hour.

  4. An IVR for spoken intents2 hr 30 min

    Build a menu-and-speech IVR for a bank's common requests, test it with open spoken-intent recordings by speaker group, and compare keyword rules, a classifier and an LLM for the intent step.

Reviews

Loading the reviews…

    Took this course? Sign in to review it. Learners can write a review after they complete 25% of the lessons.

    Share this course