The Agent Reply Quality report is AI review for your team’s own replies: every outgoing reply is scored, and anything risky is flagged — so you can coach with real examples instead of spot-checking chats by hand.
Open Reports → Agent Reply Quality. Scores appear around 30 minutes after your team replies to a conversation.
What you see
- Average reply score — overall quality from 1 (Poor) to 5 (Excellent), with a per-agent breakdown.
- Conversations scored and coverage — how much of your traffic has been reviewed.
- Flagged conversations — replies the monitor marked Critical or High, worth a look first.
- Score distribution — how your conversations spread across the five scores.
- Scored conversations — the full list, filterable by agent, inbox, team, and score, with a summary for each. Click through to read the conversation itself.
Use Download report to export the data.
Tuning what gets flagged
The Chat monitoring categories and the Agent reply scoring guidance are editable under Flow Agents → AI Monitoring: reword or switch off built-in categories, add your own business-specific rules, and mark findings as incorrect so the monitor learns to skip replies like them.
Switching a section on or off
Each section of Flow Agents → AI Monitoring has its own switch on its title line: Chat monitoring, Agent reply scoring and Lead analysis. Flip the switch, then click Save changes at the bottom of the page. Nothing changes until you save.
- Chat monitoring off — no new findings are flagged for the account. Past findings stay on the Flagged findings page.
- Agent reply scoring off — no new reply scores are added. This report keeps its history, and FAQ suggestions from your conversations keep arriving in the AI Knowledge Hub.
- Lead analysis off — conversations are no longer rated as leads, and the daily lead report is not sent. See Lead temperature.
Switch a section back on at any time and new conversations are picked up again.
This report is included on higher plans.