Session Abstract

In conversational AI, it’s unclear whether long dialogues mean users are engaged or simply stuck, and whether short ones indicate efficiency or frustration. Likes and dislikes don’t solve the mystery - most users skip feedback, and when they do, it rarely reflects the whole session. Without understanding the real user experience, teams risk optimizing blindly. Bots might look successful on paper but fail in practice, frustrating customers and wasting investment. A reliable, scalable way to interpret conversations is critical for improving both satisfaction and ROI. We’ll embed an AI prompt directly in Copilot Studio to classify user messages and configure memory to preserve conversation context. All telemetry is then streamed into Azure Application Insights as the single source of truth for behavior and performance data. From there, we extend the solution with Power Automate Process Mining, feeding in the classified dialogues to generate a DNA map and dialogue tree. This uncovers bottlenecks, reveals user paths, and enables truly data-driven decision making. Key takeaways: ✅ How to move beyond likes/dislikes and finally measure real conversational experience with AI-powered classification. ✅ How to use Azure Application Insights and Process Mining together to uncover the DNA of user journeys and pinpoint hidden bottlenecks. ✅ How to turn raw chat telemetry into data-driven decisions that improve both user satisfaction and bot performance at scale.

Katerina Chernevskaya

Katerina, an MVP, MCT, and renowned public speaker, focuses on Power Platform & AI. With twenty years of experience and key certifications in PROSCI, PMI, Power Platform Solution Architect, she leads in app development, AI, and adaptive work models. Active in the Microsoft community, Katerina mentors new tech talent and supports innovation. Katerina takes part in the #SharingIsCaring initiative, guiding community contributions to the Microsoft Sample Solution Gallery.