NTT Develops AI Technology That Visualizes Expert Knowledge for Business Succession Planning

NTT announced that it has developed the world's first AI technology capable of visualizing expert decision-making processes with approximately 90% accuracy, based on dialogue data from areas such as security incident response and call center operations. This technology addresses a longstanding challenge in query response operations: the difficulty of transferring expert knowledge.

With this technology, even less experienced staff can replicate expert-level responses. Looking ahead, NTT aims to incorporate these extracted expert decision-making processes into AI systems to enable automated responses grounded in real operational expertise, enhancing both the quality and efficiency of query handling.

This research has been accepted as a Findings paper at the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025), held from July 27 to August 1, 2025.

As labor shortages worsen in areas such as security incident response and call center operations, there is a growing need to outsource tasks to less specialized staff and to automate query handling using technologies like large language models (LLMs). Meanwhile, expert staff who play a central role in query response rely on their own unique knowledge and thought processes, making their decision-making difficult to observe or formalize. As a result, the transfer of operational knowledge in query handling has become a significant challenge.

To address this issue, NTT has developed the world's first AI technology that leverages LLM to analyze dialogue data and accurately visualize expert decision-making processes. By applying these extracted insights, even novice staff can replicate expert-level responses, enabling more consistent and efficient operations.

Using LLMs, we extract questions and suggestions from all text-based dialogue data. The extracted questions and suggestions are then grouped by content to create a merged question list and a merged suggestion list.

For each dialogue data, the LLM references the merged question list and merged suggestion list to trace which question in the list was addressed, what answer was provided, and how the conversation ultimately led to a particular suggestion.

This allows the entire process, from question to answer to suggestion, to be structured as a flow.

Applying this process across all dialogue data enables the transformation of unstructured dialogue data into structured flows.

In each structured flow, transitions from one question or suggestion to the next are treated as a single step. The LLM counts the number of times each transition occurs across all flows. These transitions are then organized into a tree structure, with frequently occurring transitions placed higher in the hierarchy. This process generates a question and decision-making flowchart that represents common response patterns.

To evaluate the accuracy of this technology, we utilized FloDial1, a publicly available dataset consisting of dialogue datasets and their corresponding flowcharts, commonly used in the assessment of natural language processing technologies. Using the dialogue data from FloDial, we generated flowcharts with our method and compared them with the ground-truth flowcharts included in the dataset.

By extracting all dialogue data from both sets of flowcharts and comparing them, we found that our method successfully reproduced approximately 90% of the question and suggestion tree structures present in the ground-truth flowcharts.

This technology addresses the challenge of transferring operational knowledge in query response tasks such as security incident handling and call center operations. By leveraging this method, even novice staff can replicate expert-level responses. Having validated the approach using FloDial, an ideal publicly available dataset, the next step is to apply it to real-world dialogue data.

To ensure practical applicability, we aim to improve the accuracy of extraction and visualization so that the system can handle incomplete data and discontinuous dialogue, which are common in real-world business contexts.

Source: NTT media announcement