Course contentsShow
← AI Engineering
Lesson 665 of 1,88616. AI Agents - ImplementationFree lesson

Task Success Metrics for Agents

Define measurable success criteria for agent tasks, such as goal completion, correctness, and efficiency.

Task Success Metrics for Agents

What You'll Learn

You'll learn how to define clear, measurable criteria to evaluate whether your AI agent successfully completes its tasks.

Why Metrics Matter

Imagine hiring a research assistant but never checking if they actually found the right information, delivered it on time, or understood your request. Without measuring success, you can't improve your agent or even know if it's working!

For AI agents, success metrics answer three critical questions:

  1. Did it complete the goal? (Goal Completion)
  2. Was the result correct? (Correctness)
  3. How efficiently did it work? (Efficiency)

Core Success Metrics

Goal Completion Rate

Did the agent reach a stopping condition with a final answer? Track:

  • Percentage of tasks that produce a final answer vs. timing out
  • Whether the agent correctly identified when to stop
  • Tasks abandoned due to errors or iteration limits

Correctness

How accurate is the agent's output? Measure:

  • Exact match against expected answers (for deterministic tasks)
  • Human evaluation scores (for open-ended tasks)
  • Hallucination rate (made-up facts or tool calls)
  • Whether the right tools were selected and used properly

Efficiency

How many resources did the agent consume? Monitor:

  • Number of reasoning iterations before completion
  • Total token usage across all LLM calls
  • Number of tool calls made (fewer is often better)
  • Wall-clock time from start to finish
  • Cost per task completion

Putting It Together

Define metrics before building your agent. For a research agent, you might set: "Success = finds 3 relevant sources (correctness), completes in under 10 iterations (efficiency), within 60 seconds (efficiency)."

Key Takeaway: Define measurable success criteria—goal completion, correctness, and efficiency—to objectively evaluate and improve your AI agents over time.