Skip to content
Français

Observability for AI Agents: Traces, Costs and Audit Trails

See what your AI agents do, what they cost and who approved each change. Instrument a live support workflow with OpenTelemetry, find agent failures in traces, split cost by model and step, and keep an audit trail that answers "why did the agent do this?".

Read every lesson as a guest, or sign in to enroll and track your progress.

Is this the right course for you?What this course covers and does not cover, and 5 courses to take for other needs.

This course teaches how to observe AI agents in production: OpenTelemetry for multi-step runs, approvals and handoffs, the failure patterns that only agents have, cost per run and per resolved ticket, and an audit trail that is complete and tamper-evident. It does not teach how to build the agent, general service telemetry, evaluation datasets or the platform controls themselves.

Take another course if you want to:

What you'll learn

By the end of this course, you will be able to:

  • Explain what to record for an agent (traces, metrics, events, audit records) and what must not be recorded, and why one tool is not enough.
  • Instrument a multi-step, tool-using agent with OpenTelemetry and its GenAI conventions, including approvals that pause a run and work split across agents.
  • Find agent-specific failures in traces (loops, hidden tool errors, duplicate writes, success without action, followed injected instructions) and prove the cause with evidence.
  • Attribute cost to runs, steps, customers and models, and set budgets and alerts that lead to a decision.
  • Keep an audit trail that is complete, tamper-evident and access-controlled, and answer an audit question from it.
  • Run the same telemetry into Grafana and Langfuse, choose tools for a team with stated criteria, and judge what a purpose-built view adds.

About this course

Larkfield’s support workflow is live. Every day it reads tickets, looks up orders and policy, proposes refunds, returns and reshipments, waits for an agent’s approval and then writes. Last week one customer was refunded twice. The dashboard stayed green, and nobody could say which step, which model decision or which person caused it.

In this course, you follow Amira, the developer who owns the workflow, as she makes it observable. You record each run as one trace with OpenTelemetry, from the ticket through the model calls, the tool calls and the approval to the write. You send the same data to Grafana and to Langfuse, and you keep a separate audit trail that nobody can change without detection.

Then you use what you recorded. You find loops, hidden tool errors, duplicate writes and injected instructions in the traces of a replayed week. You find out why the cost per resolved ticket rose after a model switch. And you answer Grace’s question: why did we refund that ticket twice?

Everything runs on your computer with no account and no key. Every model decision is a real recording from the agents course, replayed. The practice incidents are cases we caused on purpose, and each lesson says which evidence is replayed and which is simulated. Prompt text, personal data and secrets stay out of the telemetry by default, and the Collector removes them again before anything leaves your computer.

Before you start

You completed AI Agents and Workflow Orchestration, or you have the same skills. You can read and change Python, use a terminal and read JSON. The MLOps, agents governance and MCP courses help but are not required; lessons that use their ideas explain them briefly and link to them. You need no account, no key and no money. Grafana runs in Docker if you want it, and every lesson also works from saved telemetry. Langfuse is optional, in one module, with a free account.

Courses to take first: AI Agents and Workflow Orchestration.

Who this is for

AI engineers, backend developers and platform or operations engineers who run AI agents in production, and the technical leads who must say what an agent did, why, and what it cost.

Lead or use AI, but do not build it?

You can study the ideas of this course without writing code. Read modules 1, 5 and 6 (about 6–7 hours), answer their knowledge checks, and get a free concepts certificate. You can come back for the practice later.

The paths for people who lead or use AI take this course this way. See the paths

How we made this course

Experienced AI practitioners designed this course. AI helped us draft the content; people chose the teaching approach and the examples, and reviewed every lesson. The outputs that you see are the actual results of running the code. How we make our courses

Inside the course

Final module · choose at least oneCase studies

3 case studies

Three end-to-end case studies on open data. Find failures in public agent runs, measure the cost of a coding agent, and design the records that would have answered a public case. Complete at least one for the certificate; each one is a portfolio piece.

Complete at least one case study for the certificate. The others are optional. What each one covers

  1. Find failures in public agent runs150 min
  2. The cost of a coding agent150 min
  3. Could the records have answered?180 min

After the case studiesFinal assignment

Use every module in one practice project. You collect evidence, and check it against the course outcomes.

  1. Final assignment180 min

Case studies

Realistic projects from start to finish, on real data, with every output explained. Each one makes a portfolio piece. Complete at least one for the certificate; the others are optional.

  1. Find failures in public agent runs2 hr 30 min

    Convert an open set of recorded customer-service agent runs to OpenTelemetry, load them into Grafana, and find and count failure patterns with evidence.

  2. The cost of a coding agent2 hr 30 min

    Measure cost per task, per step and per outcome on open coding-agent trajectories, find the runs that loop or over-spend, and propose a budget and a step limit.

  3. Could the records have answered?3 hr

    Take a public, documented case of an AI assistant's answer that a company had to defend, list the questions it raised, and design the audit trail and traces that would have answered them.

Reviews

Loading the reviews…

    Took this course? Sign in to review it. Learners can write a review after they complete 25% of the lessons.

    Share this course