Evaluate Piper Utilizing the Scorecard & Inbox

Learn how to test and refine Piper's responses using the Scorecard and Inbox to ensure accurate and effective interactions on your website.

• Evaluating Piper's Responses with the Scorecard
• Leveraging the Question Inbox

<span id="scorecard"></span>

Evaluating Piper's Responses with the Scorecard

The Scorecard feature in the Qualified platform is crucial for testing and refining Piper's responses, ensuring that the AI SDR delivers accurate and helpful information to website visitors.

Before utilizing the Scorecard, it's important to complete the settings under the General, Content, Goals, and Rules tabs in AI Studio as it will impact the outcome of Piper’s responses within the Scorecard.

Add Commonly Asked Questions

Use the Scorecard to test Piper's understanding of your business, your desired sales outcomes, and rules of engagement. Start by adding 15-25 common questions your SDRs receive from prospects to simulate real-life interactions Piper may encounter. By analyzing Piper's responses to these questions, admins can assess the accuracy and relevance of the information provided.

Evaluating Piper's Responses

A simple feedback system within the Scorecard allows admins to evaluate Piper's responses:

• Thumbs Up: For responses that meet expectations, reinforcing correct answers.
• Thumbs Down: For unsatisfactory answers, prompting further refinement.

As responses are scored with a thumbs up or thumbs down, the AI Confidence score on the left will update to reflect Piper’s accuracy. 

Categorizing Feedback for Improvement

When providing negative feedback, categorize it to pinpoint areas needing improvement:

• Incorrect: Highlights factually inaccurate responses, prompting a review of the source material or the addition of accurate snippets.
• Not Helpful: Indicates that while the answer might be factually correct, it doesn't directly address the user's question, requiring adjustments to Piper's understanding of user intent and context.
• Wrong Tone: Addresses responses not aligning with the desired brand voice or tone, allowing for adjustments to Piper's communication style.
• Other: Serves as a catch-all for any other feedback not covered in the previous categories.

Admins may also choose to create and add relevant snippets containing the correct information or clarifying the desired response. These snippets act as additional knowledge sources for Piper, but are not weighted higher than URLs.

Review Citations and Triggered Goals 

Whether the response is correct or not, admins are able to review the content Piper used for the responses under Citations, and can easily disable content sources that may be inaccurate or outdated. Additionally, admins can see any goals that would be triggered by the question asked under Triggered Goals. 

To use the Scorecard, an Admin will go to Settings → AI Studio → Scorecard. For step-by-step instructions, check out our detailed help center article here: Testing & Refining Piper’s Responses

<span id="question-inbox"></span>

Leveraging the Question Inbox

The Inbox provides insights into questions encountered by both Piper and human SDRs over the past 30 days. It helps identify knowledge gaps in Piper's understanding and frequently asked questions, allowing admins to add crucial questions to the Scorecard for testing and refinement.

By following these best practices, you can effectively test and grade Piper, ensuring it provides accurate and helpful responses to website visitors.

To view the Inbox, an Admin will go to Settings → AI Studio → Inbox. For step-by-step instructions, check out our detailed help center article here: Testing & Refining Piper’s Responses