Session Abstract

In conversational AI, it’s unclear whether long dialogues mean users are engaged or simply stuck, and whether short ones indicate efficiency or frustration. Likes and dislikes don’t solve the mystery - most users skip feedback, and when they do, it rarely reflects the whole session. Without understanding the real user experience, teams risk optimizing blindly. Bots might look successful on paper but fail in practice, frustrating customers and wasting investment. A reliable, scalable way to interpret conversations is critical for improving both satisfaction and ROI. We’ll embed an AI prompt directly in Copilot Studio to classify user messages and configure memory to preserve conversation context. All telemetry is then streamed into Azure Application Insights as the single source of truth for behavior and performance data. From there, we extend the solution with Power Automate Process Mining, feeding in the classified dialogues to generate a DNA map and dialogue tree. This uncovers bottlenecks, reveals user paths, and enables truly data-driven decision making. Key takeaways: ✅ How to move beyond likes/dislikes and finally measure real conversational experience with AI-powered classification. ✅ How to use Azure Application Insights and Process Mining together to uncover the DNA of user journeys and pinpoint hidden bottlenecks. ✅ How to turn raw chat telemetry into data-driven decisions that improve both user satisfaction and bot performance at scale.

Katerina Chernevskaya

Katerina, rocking it as a Technical Paradigm Shifter (TPS), brings her two-decade tech saga into the Microsoft universe. Backed by stellar certifications like PROSCI Change Management Practitioner, PMI Citizen Developer Business Architect, Power Platform Solution Architect and others, she crafts slick, efficient solutions with a touch of Microsoft's finest. Mastering cloud dynamics, innovating flexible work models, and simplifying app development, she's a tech trailblazer. An active Microsoft community member and MVP, Katerina mentors emerging tech minds, keeps the innovation cycle spinning, and contributes to the Microsoft Sample Solution Gallery. In her TPS role, she's reimagining the tech landscape with her unique blend of expertise and Microsoft-centric innovation.