Skip to content
Français

Testing AI Applications: Manual and AI-Assisted QA

Find, reproduce, report and prevent the failures of AI features. You test a practice version of Larkfield's support assistant with planted bugs, write bug reports that people and coding agents can act on, and finish with a regression set and a release decision.

Read every lesson as a guest, or sign in to enroll and track your progress.

Is this the right course for you?What this course covers and does not cover, and 4 courses to take for other needs.

This course teaches you to test AI features as a tester. You explore a practice app, reproduce failures, write bug reports and keep a regression set. You do not write programs, and you do not build an automatic evaluation harness or use statistics to compare two versions.

Take another course if you want to:

What you'll learn

By the end of this course, you will be able to:

  • Explain how AI features fail differently from ordinary software, and plan risk-based tests for them.
  • Judge output when there is no single right answer, with rubrics, property checks and metamorphic tests.
  • Explore AI features systematically and measure how often a failure reproduces.
  • Write bug reports that people can triage and that a coding agent can fix, with a failing case attached, and verify the fix.
  • Test retrieval, tool-using agents and safety risks: prompt injection, data leaks, and harmful, biased and over-cautious output.
  • Use AI to assist QA work safely, and turn findings into a regression set and a release sign-off.

About this course

Larkfield’s support assistant answers customers about returns, refunds and orders. A new release is ready. Esme, Larkfield’s new QA lead, tested web shops for years. She asks the assistant the same question three times and gets three different answers. Two are fine. The third quotes a refund amount that is in no policy. Is that a bug? Can she show it to the developers again tomorrow? And what does she write so that a coding agent fixes the cause and not only the test?

In this course, you follow Esme. You test a practice version of the assistant, the bug-hunt app, which runs on your computer with no account and no key. It has planted bugs that fail always, sometimes or rarely, like real AI features. You plan by risk and judge answers when there is no single right answer. You explore with charters, measure how often a failure reproduces, and write bug reports that a developer or a coding agent can act on.

You use the words that testers already know: test case, charter, severity, regression. The course explains each AI idea when you need it. The practice app uses a simulated model, and every lesson that shows real model behaviour uses recorded runs of a real model. Each piece of evidence says which one it is.

Before you start

None. You must be able to run a command that we give you in a terminal, edit a text file and read a short JSON document. Experience of testing any kind of software helps, but the first module names the testing ideas that the course uses. You do not write programs, and you need no account, no key and no money. The practice app runs on your computer with Python.

Who this is for

Testers and test engineers, manual or automation, who must now test AI features. Developers who test their own AI features, and domain experts who join the testing of an AI product.

Lead or use AI, but do not build it?

You can study the ideas of this course without writing code. Read modules 1, 2, 4 and 6 (about 7–9 hours), answer their knowledge checks, and get a free concepts certificate. You can come back for the practice later.

The paths for people who lead or use AI take this course this way. See the paths

How we made this course

Experienced AI practitioners designed this course. AI helped us draft the content; people chose the teaching approach and the examples, and reviewed every lesson. The outputs that you see are the actual results of running the code. How we make our courses

Inside the course

Module 2ConceptsPlan and judge without one right answer

3 lessons

Plan and judge without one right answer. Put each feature on a risk grid, agree acceptance criteria with the product owner, and judge answers with rubrics, property checks and metamorphic relations instead of an exact expected text.

  1. Risk-based test planning35 min
  2. Rubrics and property checks40 min
  3. Metamorphic testing40 min

Final module · choose at least oneCase studies

3 case studies

Three end-to-end case studies on open data. Test an assistant that answers from open government guidance, rate real open-source issues for agent-readiness, and run a safety test pass with open attack sets. Complete at least one for the certificate; each one is a portfolio piece.

Complete at least one case study for the certificate. The others are optional. What each one covers

  1. Test a public-guidance assistant240 min
  2. Agent-ready reports from real open-source issues120 min
  3. A safety test pass with open attack sets120 min

After the case studiesFinal assignment

Use every module in one practice project. You collect evidence, and check it against the course outcomes.

  1. Final assignment180 min

Case studies

Realistic projects from start to finish, on real data, with every output explained. Each one makes a portfolio piece. Complete at least one for the certificate; the others are optional.

  1. Test a public-guidance assistant4 hr

    Run a full test campaign (risk table, charters, rubric and metamorphic checks, bug reports, regression set) on an assistant that answers from open government guidance, with recorded answers.

  2. Agent-ready reports from real open-source issues2 hr

    Rate real issues from an open dataset of software issues and their fixes for agent-readiness, rewrite weak ones, and check one against the fix that was merged.

  3. A safety test pass with open attack sets2 hr

    Run open prompt-injection and over-refusal sets against a recorded assistant, measure rates by category, and report the defects with severity.

Reviews

Loading the reviews…

    Took this course? Sign in to review it. Learners can write a review after they complete 25% of the lessons.

    Share this course