# Evaluating Digital Workers: Scorecards, Rubrics, and Dashboards Agentic Studio includes a built-in evaluation framework that scores every digital worker interaction after it closes. This framework lets you define what good looks like, measure whether the digital worker is meeting that standard, and identify where instructions, skills, or tools need adjustment. The evaluation approach mirrors Auto-QA: structured scorecards and rubrics produce consistent, auditable scores that roll up into dashboards for trend monitoring. ### **Scorecards** A scorecard defines what gets measured and how each dimension is weighted. It is the top-level structure that ties your evaluation criteria together. A typical scorecard includes: * A defined purpose, for example Support Quality, Dispatch Triage Quality, or Sales Qualification. * Score categories (high-level measurement buckets) such as Policy and Process Adherence, Accuracy and Resolution Quality, Communication Quality, or Safety and Compliance. * Weights for each category, for example 40% accuracy, 30% process adherence, 20% communication, 10% compliance. * One or more rubric criteria attached to each category. Once configured, a scorecard is published to one or more digital workers in a specific environment. Start with UAT before publishing to Production. ### **Rubrics** Rubrics define the specific criteria used to score an interaction. Each rubric item is a clear, observable check or question that a reviewer, human or automated, can answer consistently. Examples of rubric criteria: * Did the digital worker follow the required verification steps? * Was the final answer accurate and complete? * Did the digital worker use the correct tool or configuration? * Was the tone professional and helpful? For each criterion, you set a scoring scale (for example, Pass/Fail, 1 to 5, or 0 to 2) and add guidance for reviewers: what good looks like, common failure modes, and what evidence to look for in the conversation transcript. Rubric versions should be maintained over time so that historical interactions remain comparable and auditable even as criteria evolve. ### **Interaction-Level Results** After a conversation closes, it is scored automatically. The rubric evaluation is displayed directly on the conversation view. * Open any agent interaction to see criterion-level scores, category rollups, and the overall score. * Drill into failures to understand which rubric criteria drove the score down and what evidence in the conversation supports the rating. * Use the interaction view for spot-checking, incident review, and debugging specific changes to instructions, skills, or tools. ### **Evaluation Dashboards** Dashboards aggregate scores across interactions so you can monitor quality at scale and detect regressions over time. Dashboard capabilities include: * Trend tracking: overall score trends by day or week. * Slicing: filter by digital worker, workspace, environment, channel, score category, or individual rubric criterion. * Distribution monitoring: percentage of interactions below a threshold, pass/fail rates, and category breakdowns. * Release validation: compare scores before and after a new version is promoted to Production. Use dashboards not just to track averages, but to understand distributions. A high average score that hides a tail of very low-scoring interactions is a sign of a systematic failure mode, not overall good performance. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://kb.theloops.io/agenticstudio/resource-library/explainers/evaluating-digital-workers-scorecards-rubrics-and-dashboards.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.