Module 1Measure before scaling
4 lessonsMeasure before you change anything. Add the AI work to the ticket API, describe the workload in numbers, read averages and percentiles, and find where a request spends its time.
Scaling APIs and AI Workloads
Get an AI service ready for ten times the usual traffic, with proof. You measure Larkfield's ticket API first, then add concurrency, a safe cache, a job queue with workers, and limits that keep overload under control, and you show every change with a load test.
- 19 lessons
- 2 case studies
- 17–23 hr
- Intermediate
- Check in every lesson
- Free certificate
Read every lesson as a guest, or sign in to enroll and track your progress.
Enroll to save the lessons that you complete. It is free.
What you'll learn
By the end of this course, you will be able to:
- Estimate workload and locate bottlenecks.
- Apply concurrency, caching and queues appropriately.
- Design resilient retries and backpressure.
- Validate scaling decisions with controlled load tests.
About this course
Black Friday is coming. Larkfield’s help desk expects about ten times the usual number of tickets for a few hours, and every new ticket now starts AI work: the service classifies it, drafts a reply and computes an embedding. The AI provider takes seconds to answer and refuses requests when the help desk uses more than its quota.
In this course, you get Larkfield’s ticket API ready for that day, one measured change at a time. First you measure: you describe the workload in numbers and find the bottleneck. Then you let the service do more at once, and you see where more concurrency stops helping. You add a cache that never serves one person’s answer to another person.
Next you move the slow AI work to a queue with background workers. A repeated message still has one effect, and a full queue refuses new work with a clear message instead of growing without limit. You add timeouts, a circuit breaker and graceful shutdown, so that a slow or failed provider does not take the whole service down.
Finally you prove each change with a load test and write a capacity recommendation. Everything runs on your computer, including a simulated AI provider whose timing comes from real recorded calls. Every latency, throughput and error count in the lessons comes from a real run, and the lessons always say on which computer and under which workload.
Before you start
You completed Build and Deploy Your First API, Containers and Docker, and Databases and Storage for AI Applications, or you have the same skills. You do not need an account or money. The database, the cache, the AI provider (a simulated one) and the load generator all run on your computer.
Courses to take first: Build and Deploy Your First API, Docker and Cloud Essentials for AI Developers, Databases and Storage for AI Applications.
Who this is for
People who can build a small API with a database and run it in Docker, as in Build and Deploy Your First API, Containers and Docker, and Databases and Storage for AI Applications, and who now need it to stay fast and correct when many requests and slow AI calls arrive at once. Developers and AI engineers who own a service.
How we made this course
Experienced AI practitioners designed this course. AI helped us draft the content; people chose the teaching approach and the examples, and reviewed every lesson. The outputs that you see are the actual results of running the code. How we make our courses
Inside the course
Module 2Concurrency and service boundaries
3 lessonsDo more at once without breaking things. See how one blocking call stalls an async server, run several instances, and find out why more concurrency can give less useful work.
Module 3Caching without incorrect results
3 lessonsCache answers without serving the wrong one. Choose what to cache, put versions and the caller's identity into every key, and survive an empty or failed cache.
Module 4Queues and background workers
3 lessonsMove slow AI work to a queue. Track every job from submit to result, make a repeated message produce one effect, and refuse extra work politely before the queue grows without limit.
Module 5Resilience and autoscaling
3 lessonsStay up when a dependency fails. Use timeouts, a circuit breaker and a retry budget, scale on the right signal, and replace a worker without losing a job.
Module 6Validate and justify changes
3 lessonsProve each change with a load test. Design ramp, steady, burst and soak tests, compare configurations on percentiles, errors and cost per task, and write a capacity recommendation you can defend.
Final module · choose at least oneCase studies
2 case studiesTwo end-to-end case studies on open data. Plan capacity from a real public traffic trace, and add a queue and a cache to an open-data API under load. Complete at least one for the certificate; each one is a portfolio piece.
Complete at least one case study for the certificate. The others are optional. What each one covers
After the case studiesFinal assignment
Use every module in one practice project. You collect evidence, and check it against the course outcomes.
- Final assignment180 min
Case studies
Realistic projects from start to finish, on real data, with every output explained. Each one makes a portfolio piece. Complete at least one for the certificate; the others are optional.
Capacity plan from a public traffic trace2 hr 30 min
Model arrivals and bursts from an open request log, and compute the queue that a given number of workers would build.
Queue and cache for an open-data API4 hr
Put an open-data API under load, add a cache and a background export job, and measure what each change does.
Reviews
Loading the reviews…
Took this course? Sign in to review it. Learners can write a review after they complete 25% of the lessons.