Session Abstract

In conversational AI, it’s unclear whether long dialogues mean users are engaged or simply stuck, and whether short ones indicate efficiency or frustration. Likes and dislikes don’t solve the mystery - most users skip feedback, and when they do, it rarely reflects the whole session. Without understanding the real user experience, teams risk optimizing blindly. Bots might look successful on paper but fail in practice, frustrating customers and wasting investment. A reliable, scalable way to interpret conversations is critical for improving both satisfaction and ROI. We’ll embed an AI prompt directly in Copilot Studio to classify user messages and configure memory to preserve conversation context. All telemetry is then streamed into Azure Application Insights as the single source of truth for behavior and performance data. From there, we extend the solution with Power Automate Process Mining, feeding in the classified dialogues to generate a DNA map and dialogue tree. This uncovers bottlenecks, reveals user paths, and enables truly data-driven decision making. Key takeaways: ✅ How to move beyond likes/dislikes and finally measure real conversational experience with AI-powered classification. ✅ How to use Azure Application Insights and Process Mining together to uncover the DNA of user journeys and pinpoint hidden bottlenecks. ✅ How to turn raw chat telemetry into data-driven decisions that improve both user satisfaction and bot performance at scale.

Katerina Chernevskaya

🏅 Microsoft MVP, MCT, and Power Platform Solution Architect with 20+ years in tech. 🏗️ Builds enterprise solutions with Power Platform, Copilot Studio, Microsoft Foundry, and Azure AI. 🤖 Particularly interested in the moment an AI agent stops answering questions and starts doing things - and the architecture needed to keep that under control. 🎤 Speaks internationally about AI, governance, and the messy gap between a great demo and a production system. 🧪 Mentors new tech talent and believes the best way to learn something is to build it, break it, and share what happened. 📜 Collects certifications too: Power Platform, Azure AI, Prosci, and PMI.