Skip to content
Français

A hosted speech API

  • Lesson 12
  • 25 min
  • Module 4: Speech recognition: compare the families

Where we are

We measured local recognisers on our laptop. Many teams send their audio to a hosted service instead. In this lesson, we look at what a hosted speech API adds, what it costs, and where the callers' audio goes. First, think of an office that sends its recordings out to be typed.

What you will be able to do

Decide when Larkfield should send its call audio to a hosted speech service, and when it should keep recognition on its own computers. The decision depends on cost, features and, most of all, on where the callers’ voices go.

Before you startWhat to know and to have readyAbout 25 minutes

What you need to know

What you need

  • Nothing to install. This lesson runs no code. The course team did not send any audio to a hosted service: every number here is either measured on our laptop or a price that a vendor publishes.
  • For the optional live mode at the end only: your own account with a speech service, and your own key. It costs money. Skip it if you do not have one.

Before computers could recognise speech, many offices recorded meetings on tape. Then they posted the tapes to a typing agency. The agency had many typists, so a big pile of tapes came back quickly. It charged by the minute of audio. The office did not need its own typists, desks or training.

But the office also had to ask some questions. Who at the agency listens to our tapes? Do they keep a copy? Where is their office, and which country’s law applies there? And what if the agency changes its typists and the quality drops?

A hosted speech API is that agency, as a web service. An API (application programming interface) is a way for one program to call another. Your program sends audio over the internet to the vendor’s servers. The vendor’s recogniser writes the text and sends it back. You pay for each hour of audio.

The problem: Larkfield does not want to run servers

Section titled “The problem: Larkfield does not want to run servers”

In the last lesson, Whisper small ran on a laptop CPU. Larkfield’s 4,000 calls a month are about 49 hours of audio (4,000 × 43.9 seconds). Someone must run that recogniser every day, update it, watch it, and fix it when it stops. Omar, the product manager, asks: “Can’t we just pay a company to do it?”

Yes, they can. And for some teams it is the right choice. But a hosted service changes three things: what you get, what you pay, and where the audio goes. We look at each one.

The big speech services offer more than a transcript. These are the features that vendors list most often. Each vendor has its own set, so check the documentation of the one you consider:

  • Scale without servers. Send a thousand hours at once, and the vendor runs them in parallel.
  • Word times and confidence. Each word comes with its start and end time and a score of how sure the model is. Local Whisper can also give word times: Module 5 uses them.
  • Speaker labels (diarization). “Speaker 1” and “Speaker 2” on a mono recording. Module 5 builds this locally.
  • A custom vocabulary. A list of phrases, such as product names, that the service should prefer. The next lesson does this with Whisper’s prompt.
  • Language detection and many languages.
  • Redaction. Some services can remove card numbers or names from the text. Module 9 builds this, for the text and for the audio.
  • Real-time streaming. Text arrives while the caller speaks. A phone line that answers callers needs this.
  • A service agreement. The vendor promises a level of uptime, and you have someone to call.

None of these is magic. Each one is also possible on your own computers, with work. The question is whether that work is worth more than the price.

Vendors publish a price per hour or per minute of audio. These are vendor claims: list prices for paying as you go, in one region, without discounts or contracts. We read them on the vendors’ own price sources. Prices change: check the current page before you decide.

Service What it is Published price per audio hour
Azure AI Speech, batch transcription send files, get text later US$0.18
Azure AI Speech, fast transcription send a file, get text in seconds US$0.36
Azure AI Speech, real time text while the person speaks US$1.00
Amazon Transcribe, batch send files, get text later US$0.36 (US$0.006 a minute)
Amazon Transcribe, streaming text while the person speaks US$0.60 (US$0.01 a minute)

The Azure prices are for the East US region, from the Azure AI Speech price page and Microsoft’s own price list behind it. The Amazon prices are for US East, from the examples on the Amazon Transcribe price page. Other vendors, such as Google Cloud and OpenAI, also sell speech to text. This lesson does not quote their prices: check their own pages.

A tiny worked example: one month of Larkfield’s calls

Section titled “A tiny worked example: one month of Larkfield’s calls”

Local recognition also has a cost per audio hour. In Whisper sizes on a laptop, Whisper small had a real-time factor of 0.075. The course assumes that a CPU server (4 cores, 16 GB) costs US$0.17 an hour, which is an assumption for a typical cloud price. So one hour of audio costs 0.075 × US$0.17 = about US$0.013 of compute.

For Larkfield’s 49 hours a month:

Way to run it How the cost is found Cost a month
Whisper small, server started only for the work 49 h × 0.075 = 3.7 h of compute × US$0.17 about US$0.62
Whisper small, server always on 730 hours a month × US$0.17 about US$124
Azure batch transcription (vendor price) 49 h × US$0.18 about US$8.78
Azure real time (vendor price) 49 h × US$1.00 about US$48.78

Look at the second row. The cost per audio hour is cheap only if you pay for the server just while it works. A server that waits all month for 49 hours of audio costs more than the hosted batch service. At Larkfield’s volume, every option costs less than one person’s working day. The cost of the computer is not what decides here. The errors and the privacy decide.

  • local Whisper
  • hosted (vendor price)
US cents per hour of audio. Local: measured real-time factor × US$0.17 an hour for a CPU server (an assumption). Hosted: the vendors' published prices (vendor claims; prices change). B: batch; RT: real time.
Text description of the chart

Five columns, in US cents for one hour of audio. Two local Whisper sizes: tiny 0.3 cents and small 1.3 cents (medium, not shown, is 3.3 cents). They are so small that they almost disappear next to the hosted prices: Azure batch 18 cents, Amazon batch 36 cents and Azure real time 100 cents (one dollar). Amazon streaming, 60 cents, is in the table above. The local numbers count only the compute time, not an idle server, your team’s time or the updates.

This is the most important part. A call recording is a recording of a person’s voice. It often holds a name, an address, an order, and sometimes a card number said aloud. When you use a hosted service, that recording leaves Larkfield.

A flow chart. The call recording on Larkfield's phone system has two paths to the transcript in Larkfield's database. The local path: Larkfield's server runs Whisper on the CPU. The hosted path, the optional live mode: the internet, an encrypted upload with your key; the vendor's region, the speech service, which sends the transcript back; and a dotted line from the speech service to the vendor's storage and logs, with the questions how long and who reads.
Two paths for the same recording. On the local path, the audio never leaves Larkfield. On the hosted path, it travels to the vendor's region, and the vendor's storage and logging rules apply to it.
Text description of the diagram

The call recording starts on Larkfield’s phone system. On the local path, Larkfield’s own server runs Whisper, and the transcript goes into Larkfield’s database. The audio never leaves Larkfield. On the hosted path, the recording goes over the internet, encrypted, with your key. It arrives in the vendor’s region, where the speech service makes the transcript and sends it back to Larkfield’s database. A dotted line goes from the speech service to the vendor’s storage and logs. That line has two questions: how long is the audio kept, and who can read it.

Before Larkfield sends one call, someone must answer these questions, in writing, from the vendor’s own terms:

  1. Does the vendor keep the audio or the text, and for how long? Some services delete them after processing. Others keep them for a time, for example for abuse checks.
  2. Does the vendor use the data to improve its models? Many business services say no, but read the terms for your plan.
  3. In which region is the audio processed and stored? Laws such as Europe’s GDPR care about where personal data goes.
  4. Did the callers agree? Larkfield’s recording notice says the call is recorded “for quality and training”. Does it cover sending the recording to another company?
  5. What about card numbers? A card number said aloud must not travel to more places than it must. Module 9 removes it from the text and the audio.

The AI security course teaches how to judge these questions. For this course, remember the rule: the local path keeps the audio inside Larkfield; the hosted path makes it someone else’s question too.

When to use a hosted service, and when not

Section titled “When to use a hosted service, and when not”
Use a hosted service when Keep it local when
You need many languages or features that you cannot build soon The audio holds personal data you cannot send out, or your callers did not agree
Your volume jumps up and down, and idle servers would cost more Your volume is steady, so a server is busy most of the time
You have no one to run and update a model You need the same model for months, so your measurements stay true
You have measured the service on your own calls You have not measured it yet

That last row matters. A vendor’s accuracy claim is measured on the vendor’s test audio, not on Larkfield’s phone line. And a hosted model can change without a new version number on your side. If Larkfield chooses a hosted service, it must measure it on the same test calls, with the same normalisation guide, before it decides. Then it measures it again from time to time.

The course does not run a hosted recogniser. You can, if you have your own account. It costs money, and the course team did not test these steps. Do this only with Larkfield’s synthetic calls, never with recordings of real people.

The 60 test calls are 2,834 seconds of audio, about 0.79 hours. At Azure’s published batch price, that is about US$0.14. Check the price on the vendor’s page first.

  1. Create a speech resource in your vendor’s console. Choose a region, and read the data terms of your plan.

  2. Put the key in an environment variable, for example SPEECH_KEY. Never write the key in your code or in a notebook that you share.

  3. Follow the vendor’s own quickstart to transcribe one file. Start with data/calls/wideband/C-0225.flac or the phone file data/calls/phone/C-0225.wav. Some services want a WAV file at 16,000 samples a second, so you may need to convert the FLAC file first.

  4. Transcribe the 60 test calls. Write one line per call to data/recordings/asr/hosted-mine.jsonl, in the same form as the course’s recordings: {"call_id": "C-0009", "kind": "phone", "text": "...", "seconds": 0}. Put the time you waited in seconds, if you have it.

  5. Score it with the same command as every other recogniser: speech score --method hosted-mine --by language. Now your row compares fairly with the course’s table.

  6. Delete the resource when you finish, so it costs nothing more. Check the vendor’s console to see what it stored.

Common mistake

Comparing the vendor's number with your own

Symptom: Tomás reads a vendor page that says “word error rate under 5%”. Our Whisper small has 8.6% on the phone test calls. “The hosted service is clearly better,” he says.

Cause: the two numbers come from different audio, different words and different normalisation rules. The vendor’s 5% may be on clear read speech at 16,000 samples a second. Ours is on an 8,000-samples-a-second phone line, in English and French, with a written guide. In Normalise before you compare, only the normalisation changed Whisper small from 18.1% to 8.6% on the same transcripts.

Fix: a number counts only on your own test calls, with your own guide. Use the live mode above, then speech score.

Consequence in AI work: a team that buys on a vendor’s number finds the gap in production, after the contract is signed.

Check your understanding

Larkfield sends about 49 hours of calls a month. Using the prices in this lesson, what does one month cost with Azure batch transcription, and with Whisper small on a server that runs only for the work? Why is the price not the deciding factor?

Show the answer

Azure batch: 49 × US$0.18 = about US$8.80 (a vendor price; check the current page). Whisper small: 49 × 0.075 = about 3.7 hours of compute × US$0.17 = about US$0.62 (measured real-time factor × an assumed server price). Both are tiny next to Larkfield’s call costs, such as US$4.00 for each call that a person handles. So the decision rests on the errors on Larkfield’s own calls, and on where the audio may go.

Check your understanding

Name three questions Larkfield must answer, from the vendor's terms, before it sends one call recording to a hosted speech service.

Show the answer

Any three of these. Does the vendor keep the audio or the text, and how long? Does it use them to train its models? In which region is the audio processed and stored? Do Larkfield’s callers agree that their recording goes to another company? How are card numbers said aloud kept out? Answer them in writing, from the terms of the plan that Larkfield will buy.

Check your understanding

A vendor's page says its WER is 4%. How do you find out what it would be for Larkfield?

Show the answer

Transcribe the same 60 test calls with the service, save the transcripts in the course’s form, and score them with speech score. Then the same calls, words and normalisation guide stand behind both numbers. Score by slice too (language, line), because a good overall number can hide a bad slice. Use only synthetic calls for this test.

Recap

  • A hosted speech API is a recogniser on a vendor’s servers. You send audio and get text back, and you pay per audio hour.
  • It adds scale, word times, speaker labels, custom vocabulary, language detection, redaction, streaming and a service agreement.
  • Published prices are vendor claims that change: here, Azure batch US$0.18, Amazon batch US$0.36 and Azure real time US$1.00 an hour. Local Whisper small costs about US$0.013 an hour of compute, but only if no server waits idle.
  • At Larkfield’s volume, the computer is cheap either way. The errors and the privacy decide.
  • The hosted path sends callers’ voices to another company: check retention, training use, region, consent and card numbers first.
  • Never compare a vendor’s WER with yours. Measure the service on your own test calls.

You now know what a hosted service adds and what it asks of you. But both Whisper and the hosted services miss Larkfield’s product names and order numbers. Next, you give the model Larkfield’s own words, and you correct order numbers with rules.

Next: Help the model with your words