Roundtable Discussion: Automating Voice of Customer Analysis with AI Workflows Built on Generic GPT-40-Mini Model

Traditional Voice of Customer (VOC) data analysis methods are labor-intensive and time-consuming, often requiring hundreds of hours for qualitative analysis.

This discussion introduces an AI-powered workflow based on commercially available large language models that automates the most laborious steps of VOC qualitative data analysis, reducing processing time from hundreds of hours to minutes while maintaining acceptable accuracy and improving consistency. The proposed workflow leverages the generic GPT-4o-Mini model to process and prepare transcript data, identify customer need statements, and organize these statements into a hierarchical structure for further quantitative analysis. The workflow includes stages such as transcript preparation, chunking text for analysis, identifying and scoring needs, and clustering needs into prioritized categories. Evaluation results demonstrate a significant reduction in time and effort, with the AI workflow achieving a 99% reduction in analysis time compared to traditional methods with sufficient specificity of customer need if the chunk sizes are adequately small. Furthermore, the results help to overcome several typical cognitive and motivational biases of the analysts. The findings highlight the potential of AI to enhance the efficiency and consistency of VOC analysis, providing valuable insights for product development and customer satisfaction improvement.

While the discussion will mainly be presentation and Q&A, we will try to open it up in general for lessons learned in using AI for qualitative analysis of documents. At the end, and if there is time, we will discuss the current work building on these lessons-learned. We have set up AI workflows on generic LLMs for semi-automating threat modeling to develop security requirements for custom systems.