Agent Evals

Thu, 20 Aug 2026 00:54:58 +0545

notes from this short video.

We want agents to be reliable. Reliability = Observability + Evals

Observability means we can see what agent did. What the model response was, what the tool response was.

LLM response desired properties:

Relevance: Answer relevance to the user’s query
Groundedness (relates to hallucination as well): uses the particular context provided (as opposed to the generic response)
Tool Sequence Accuracy: does not do wasteful tool calls, and follows correct sequence of tool calls

Evals on aayushprime