In the quest to understand their customers better, companies are increasingly turning to generative AI tools. These advanced technologies, combined with the reasoning capabilities of large language models (LLMs) are transforming how businesses access and analyze customer insights. This shift is particularly evident in large organizations where employees often struggle to locate and understand the wealth of available data.
Customer insights come from a variety of sources, including market research, sales interactions, social media, and customer service tickets. The challenge lies in summarizing, categorizing, storing, and accessing this vast amount of structured and unstructured information. Generative AI offers a powerful solution to these knowledge management obstacles.
The Evolution of Knowledge Management
Historically, knowledge management tools like Lotus Notes and Microsoft SharePoint provided broad access to customer insights. However, these tools often failed to deliver a revolution in insights access and usage due to cultural challenges such as difficulty in organizing and retrieving knowledge, indifference to content, organizational silos, and lack of collaboration with external agencies. Today, organizations still face these same cultural obstacles.
To overcome these challenges, companies are adopting retrieval-augmented generation (RAG) a technique that integrates a company’s own customer insights with the general knowledge base on which an LLM was trained. This hybrid approach enables employees to access and summarize content in natural language, making it particularly valuable in large organizations.
How Generative AI Tools Enhance Customer Insights
Companies can leverage generative AI tools in various ways. Some opt to combine these tools with their own customer and market insight content, while others rely on external software vendors to simplify the process. For instance, Procter & Gamble (P&G) uses vendor-supplied software for knowledge storage and access but has developed its own system for generative AI-based analysis and categorization of content. This approach allows P&G to obtain sharp, pointed answers from generative AI rather than just links to documents.
Tool vendors offer different focuses, with some concentrating on the storage and access of insights, while others emphasize the analysis of qualitative customer data. The value of these tools extends beyond simple automated virtual filing cabinets. They include automated curation of documents, integration of diverse content types, on-demand analysis, and synthesized answers to prompts. However, these tools assume that data analysis has already been done and that insights are waiting to be found.
Novartis’s Sherlock System
Novartis offers a compelling example of a company successfully revamping insight storage and access using generative AI tools. In collaboration with an external vendor, Novartis developed a customer and market insights system called Sherlock. This system allows users to pose questions and receive answers by pointing to specific lines of text or timestamps in videos. Sherlock also incorporates expert-curated microsites, known as Knowledge Zones, on particular topics, such as packaging. Users must adhere to strict governance guidelines about document formats and quality, ensuring the system’s effectiveness.
The Sherlock system has helped Novartis avoid redundant insights services across its business and has enabled employees to find relevant insights quickly. The company saved over $29 million in primary market research costs in just one year, demonstrating the significant benefits of generative AI in enhancing insight storage and access.
Qualitative Data Analysis: A Special Challenge
Qualitative data analysis has long been a complex and time-consuming task. Historically, market researchers relied on semi-manual analyses using spreadsheets. Generative AI tools offer an alternative to this labor-intensive process, providing a more efficient and rigorous approach to analyzing qualitative data.
Tracy Tuten, who leads qualitative research at market research agency Illuminas (now part of Radius Insights), became an early adopter of a vendor’s generative AI software. This tool allows her to upload audio and video files for automatic transcription, generate summaries, surface themes, and compare them across audience segments. A large-scale qualitative project that might have taken six weeks to analyze in the past can now be synthesized in a day, significantly improving efficiency and rigor.
Despite these advancements, it’s important to note that generative AI does not replace researchers; it augments their performance. Uncritical use of generative AI may have significant shortcomings, and human oversight remains crucial.
Challenges and Considerations
While generative AI tools offer numerous benefits, they cannot replace human strategic thinking. Stephan Gans, senior vice president and chief customer insights and analytics officer at PepsiCo emphasized that raising the bar on marketing and innovation effectiveness will become increasingly automated. However, leading the understanding of consumer demand remains a strategic task that requires human input.
Several factors inhibit the ability of AI technology to transform customer and market insights. These include geographical and business unit disparities, the lack of integration of customer insights into strategy and culture, and the complexity of agency relationships. Addressing these challenges is crucial for companies to fully leverage the potential of generative AI in customer insights.
By overcoming traditional knowledge management obstacles and leveraging advanced technologies, businesses can gain deeper understanding of their customers and make more informed decisions.
