Latest / 1 More Paper / Real-Time Conversational Analysis Using LLMs for B2B E-Commerce Customer Value Management — Nugroho et al. (2026)
Transcript
- You’re listening to “Real-Time Conversational Analysis Using LLMs for B2B E-Commerce Customer Value Management,” by I.A. Nugroho, K. Adi, and K.B. Aryasa. Published in 2026. Abstract. The integration of artificial intelligence into business communications has created new opportunities for real-time analysis of customer interactions. This study presents an integrative framework that combines Large Language Models (LLMs) with Customer Value Management (CVM) to enhance the understanding and management of Business-to-Business (B2B) customer relationships. The proposed framework is designed to process conversational data directly, analyze its linguistic and emotional aspects, and map the results into customer value indicators, such as the Customer Satisfaction Index (CSI), the Customer Lifetime Value (CLV), and the Customer Retention Probability (CRP). The system architecture has four main layers, namely Input, Processing, Integration, and Decision layers, which convert conversational data into actionable business insights. The Random Forest (RF) classifier achieved 91% accuracy in customer segmentation (Loyal, At-Risk, Critical), while the Logistic Regression (LR) model for customer retention prediction had an AUC of 0.83. Therefore, it can be concluded that integrating LLM-based conversational intelligence with CVM metrics can enhance the accuracy of analysis and improve decision-making effectiveness. The findings of this research contribute to the development of AI-based customer value analytics by bridging natural language understanding and quantitative business intelligence. The proposed approach provides a scalable and interpretive solution to improve proactive B2B customer engagement and support adaptive decision-making systems in the data-driven enterprise era. Keywords-large language models (LLMs); real-time conversational analysis; customer value management (CVM); business-to-business (B2B); customer retention prediction I. INTRODUCTION. In the era of the digital economy, Business-to-Business (B2B) interactions increasingly rely on real-time communication channels such as live chat, virtual meetings, and digital conversation platforms. These channels generate large amounts of unstructured conversational data that, when analyzed effectively, can provide strategic insights into customer expectations, satisfaction levels, and potential long-term value. However, traditional Customer Value Management (CVM) methods still rely heavily on retrospective surveys, transaction logs, and lagging performance indicators, which are often insufficient for capturing the dynamic and nuanced nature of B2B relationships. This gap underscores the need for intelligent systems capable of analyzing real-time conversations to produce adaptive, predictive, and actionable insights. Recent advances in Large Language Models (LLMs), such as GPT, BERT, and LLaMA, have significantly transformed natural language processing by enabling deep semantic understanding, emotion detection, intent recognition, and contextual reasoning at scale. When applied to B2B conversational data, LLMs offer organizations a more contextual and dynamic understanding of customer behavior, enabling more accurate assessments of customer needs, early identification of value-creation opportunities, churn prevention, and personalized engagement strategies. Real-time conversation analytics further enhances B2B CVM by allowing continuous monitoring of emotional tone, sentiment shifts, and communication patterns, while providing automated recommendations that strengthen proactive decision-making. A growing body of research has explored the use of Artificial Intelligence (AI) in customer analysis and business interactions. In, text mining was used with LASSO and Decision Trees (DT) to identify key satisfaction drivers in fast-food customer reviews, although this work remained in the B2C domain and lacked real-time analysis. In, a fine-tuned BERT-based model was developed for emotion detection, achieving deeper contextual understanding, yet still operating on static text inputs. Complementing these findings, in, it was shown how emotional cues in online interactions influence customer engagement and purchasing behavior, while in, a multimodal voice-language model was proposed for automatically assessing customer satisfaction. In the context of LLM-based analytics, in, the effectiveness of GPT, BERT, LLaMA, and RoBERTa was demonstrated in identifying user intentions within tourism chatbots. In, BERT-based models were optimized for sentiment and emotion detection. In, an LLM-based behavior agent was capable of generating contextual, personality-driven dialog patterns through prompt engineering—highlighting the potential of LLMs to maintain conversational coherence, although this work focused on NPC communication. In, an AI-based customer experience enhancement agent was developed for the telecommunications sector, and in, the role of NLP, chatbots, virtual assistants, and speech recognition was examined in improving customer experience. These studies collectively show the increasing adoption of AI techniques—ranging from text mining and deep learning to multimodal fusion—to better understand customer perceptions, emotions, and behavior. Despite these advances, most previous research focused on static B2C scenarios or specific service domains and does not explore how real-time conversational insights can be systematically linked to strategic CVM indicators such as Customer Satisfaction Index (CSI), Customer Lifetime Value (CLV), and Customer Retention Probability (CRP). This limitation reveals a clear research gap that has not yet been addressed. This study proposes a novel integration of LLM-powered real-time conversation analytics within the CVM framework for B2B environments. By combining recent advances in NLP with B2B relationship management strategies, this research contributes both theoretically to AI-supported business intelligence and practically to enhancing customer value through contextual, predictive, and personalized analytical methods. II. METHODOLOGY. This research used an experimental and conceptual design approach to develop an integrative framework that combines LLMs with CVM. This study focused on three main areas: (i) AI-based conversation analysis, (ii) integration with CVM metrics, and (iii) predictive decision-making support in B2B business relationship management. Figure 1 illustrates the framework of the proposed method, which includes four main components: Input, Processing, Integration, and Decision layers. A. Input Layer The Input layer serves as the initial point for all data that the system will analyze. This layer manages the collection, filtering, and standardization of various customer interaction data, such as text, audio, and conversation transcripts. Its purpose is to ensure that the data is relevant, clean, and properly structured so that the LLM can read and process it effectively in the next stage. B. Processing Layer This layer is the core of the system that performs semantic, emotional, and contextual conversation analysis using LLMs. In this stage, the preprocessed data is analyzed to identify conversation intent (intent detection), emotional polarity (sentiment analysis), and relationships between linguistic entities (semantic relation extraction). LLMs such as GPT, BERT, or LLaMA are used to deeply understand the meaning and nuances of communication in single sentences or ongoing conversations. This process also involves tokenization, embedding, and contextualization of text, enabling the system to understand real-time conversational dynamics between customers and service providers. Therefore, this layer is crucial for converting raw data into meaningful linguistic representations that can later be mapped to customer value indicators (CVM) during the subsequent integration stage. For reproducibility, several details of the LLM configuration were standardized. The models used a maximum sequence length of 256 tokens, a batch size of 16, and default learning-rate parameters from their pre-trained checkpoints. BERT-base and LLaMA-2 were used in zero-shot or few-shot inference mode without additional fine-tuning to maintain model generality, while sentiment and intent thresholds were set at 0.5 to ensure consistent classification boundaries across models. GPT-4 was accessed via the OpenAI API using standard prompts without any model fine-tuning. This configuration ensures reproducibility across environments and complies with the privacy and confidentiality requirements. C. Integration Layer This layer links the results of the LLM-based conversation analysis with the CVM framework. Outputs from the Processing layer, such as emotion detection, sentiment intensity, and conversational intent, are converted into quantitative features that can be associated with customer value indicators, such as Customer Satisfaction Index (CSI), Customer Lifetime Value (CLV), and Customer Retention Probability (CRP). This process employs statistical models or machine learning algorithms to connect linguistic analysis results with business metrics, allowing contextual and predictive evaluations of customer value. This layer helps bridge NLP with strategic customer value measurement and improves the system's relevance for decision-making in B2B relationships. D. Decision Layer This layer represents the final stage of the proposed framework, converting data integration results into strategic insights and flexible recommendations for business decisions. It also interprets outputs from the Integration layer, such as satisfaction scores, retention potential, and customer value improvement opportunities, into actionable intelligence. Using predictive models and AI-driven recommendation systems, this layer provides specific suggestions, such as strategies to enhance customer relationships, segmentation priorities, or personalized interventions based on B2B customer traits. Furthermore, this layer can feature visual analysis through interactive dashboards or automated reports, making it easier for management to monitor real-time customer value performance. III. RESULTS AND DISCUSSION. The proposed model was tested and validated to assess its effectiveness and reliability. This section focuses on the results of data collection, quantitative and qualitative analysis, and a discussion of the model's performance in the context of real-time conversation analysis in a B2B environment. A. Input Layer This study used B2B customer conversation data obtained from a restricted-access e-commerce platform, provided specifically for research purposes with the permission of the platform owner and in compliance with data protection policies. The dataset consists of customer interactions collected between October 2024 and March 2025, totaling more than 91.234 conversational records that represent the dynamics of communication between business customers and service providers. Data included conversation transcripts from customer support systems (customer service logs), chatbot interactions, and business email correspondence automatically recorded by company systems. Each conversation was analyzed based on the context and theme of the communication, then grouped into several main categories: payments and finance, technical issues and platform performance, delivery and ordering of goods, account management and tax documents, after-sales service, and other topics related to informative or administrative aspects. Table I describes the data collected. B. Processing Layer The Processing layer is the core of the system, responsible for analyzing the semantic, emotional, and contextual aspects of each customer conversation using LLMs. In this stage, three main models—BERT-base, LLaMA 2, and GPT-4—are used to evaluate the system's ability to recognize conversational intent (intent detection), analyze sentiment polarity, and identify customer engagement levels. Before performing analysis, conversational data must be preprocessed, including text cleaning, tokenization, and normalization of linguistic context, so that each model can effectively read and understand sentence structure. Subsequently, the inference process runs in real time to evaluate customer emotions and communication direction. Table II shows the evaluation results for the BERT-base, LLaMA 2, and GPT-4 models. The results of the three models reveal notable differences, particularly in their ability to understand the context of customer conversations in B2B scenarios. In general, GPT-4 achieved the highest performance with an accuracy of 0.94, a precision of 0.96, a recall of 0.95, and an F1-score of 0.93. These values indicate that GPT-4 can classify the sentiment and intent of customer conversations with a very low error rate. This proves that GPT-4 excels in capturing semantic nuances and contextual relationships between sentences, which is crucial in complex and multi-layered B2B communications. The LLaMA 2 model is in second place with competitive performance (accuracy 0.89, precision 0.93, recall 0.91, and F1-score 0.90). This model has a balance between computational efficiency and analytical precision, making it suitable for systems with limited resources but requiring fast analysis. LLaMA 2 can adapt well to different customer language styles, although it is not as precise as GPT-4 in understanding ambiguous conversational contexts. Meanwhile, BERT-base showed the lowest performance among the three, with an accuracy of 0.86 and an F1-score of 0.86. However, this model consistently demonstrated stable results and reliability in the interpretation of short or simple structured conversation texts. The drawback of BERT-base is its failure to retain long-range dependencies in conversational contexts, often losing meaning in multi-utterance dialogs. Figure 2 shows a visualization of the performance evaluation results of the three LLMs. GPT-4 has consistent performance across all metrics, while LLaMA 2 strikes a balance between accuracy and computational efficiency, and BERT-base provides stable results with fewer resources. Based on the evaluation results, it can be concluded that GPT-4 is the most optimal model for implementing AI-based real-time conversation analysis systems in B2B environments. This model not only offers high accuracy and precision but also excels at identifying communication intent with better contextual sensitivity than other models. Meanwhile, LLaMA 2 provides an efficient alternative with performance near GPT-4, and BERT-base remains relevant for lightweight analysis or systems with limited computational resources. C. Integration Layer In this stage, the output from the Processing layer, which includes sentiment values (positive, neutral, and negative), conversation intent, and customer engagement levels, is transformed into numerical features that can be used to predict customer value contextually. This process integrates statistical models and machine learning algorithms to establish the relationship between linguistic variables derived from LLM analysis and relevant business indicators. The indicators used in this study include three main dimensions: CSI, which measures the level of customer satisfaction based on consistent positive sentiment and successful conversation resolution, CLV, which estimates the long-term economic value of customers based on the frequency and intensity of positive interactions; and CRP, which evaluates the likelihood of customers maintaining business relationships or making repeat purchases within a certain timeframe. Figure 3 presents the results of the mapping between sentiment polarity and customer value indicators. The customer sentiment distribution is highly uneven, with the neutral sentiment at 96%, positive at only 3%, and negative at 1%. This distribution pattern describes the characteristics of B2B communication, which tends to be formal, informative, and more focused on problem-solving than on extreme emotional expression. Customer interactions mostly occur in a stable professional context without significant emotional changes. Based on customer value indicators, neutral sentiment shows the highest value in all three key metrics, with a CSI of 0.91, a CLV of 0.87, and a CRP of 0.81. This shows that although neutral conversations are not emotional, it turns out that consistent, polite, and responsive interactions create stronger perceptions of satisfaction and trust among B2B customers. In other words, stability in communication is a key factor in maintaining long-term customer value. Interactions with positive sentiment, which only accounted for 3%, still made a significant contribution to customer relationships. Despite having a lower score than neutral sentiment, the CSI (0.83), CLV (0.80), and CRP (0.76) scores showed that communications expressing appreciation or satisfaction strengthened customer loyalty and enhanced positive views of the company's performance. This suggests that positive emotional expressions, although infrequent in a B2B setting, can significantly influence customer retention and economic value. Conversely, interactions with negative sentiment, despite making up just 1%, lead to a significant decline in all customer value metrics, with CSI at 0.68, CLV at 0.65, and CRP at 0.59, showing that despite the small volume of negative conversations, their impact on customer value remains very significant. In customer relationship management, this highlights the significance of real-time monitoring systems and LLM-based early warning tools to identify potential customer complaints immediately and respond proactively before loyalty declines or customer churn occurs. In general, these results indicate that customer value is affected not only by communication volume but also by the quality and tone of the interactions. Neutral but stable and professional conversations have been shown to contribute more to increased customer satisfaction and loyalty than extreme emotional expressions. This aligns with the idea that, in a B2B context, the success of customer value management is more dependent on the consistency of service interactions than on the frequency of positive emotional expressions. D. Decision Layer This layer not only presents the analysis results but also offers an intelligent decision-making mechanism that supports data-driven customer relationship management. In this stage, the CSI, CLV, and CRP values generated from the Integration layer were further processed using machine learning algorithms to build customer behavior classification and prediction models. This study combined the Random Forest (RF) classifier and the Logistic Regression (LR) model to produce a comprehensive and interpretive analysis. The RF model was employed to classify customers into three strategic segments, Loyal, At-Risk, and Critical, based on their CSI and CLV scores. Simultaneously, the LR model was applied to predict CRP using sentiment patterns and interaction frequency derived from the LLM analysis. This combined approach offers benefits in both classification accuracy and interpretability, allowing the system to deliver more adaptive and context-aware recommendations for the dynamics of customer relationships in a B2B business environment. Table III shows the classification results of the RF model for customer segmentation. The F1-score of 91% indicates a high level of precision in classifying customer data characteristics. More specifically, the Loyal segment has the highest precision value of 0.93, meaning that most of the customers classified as ''Loyal'' fall into that category. A recall of 0.91 and an F1-score of 0.89 suggest the model reliably detects loyal customers. In the At-Risk segment, a precision of 0.89 and a recall of 0.86 show the model is quite effective at identifying customers likely to churn, despite some misclassifications between the At-Risk and Critical categories. Meanwhile, the Critical segment shows a precision of 0.91 and a recall of 0.87, indicating the model's strong ability to identify high-risk customers. However, the slightly lower F1-score of 0.82 highlights challenges in classifying this group due to behavioral pattern similarities with the At-Risk segment. Overall, the performance of the RF model demonstrates a good balance of precision, generalization, and its ability to effectively capture customer relationship dynamics through customer value metrics, such as CSI and CLV. Figure 4 shows the confusion matrix of the RF model, which illustrates the relationship between the actual and predicted classes of the model for each customer segment. The subsequent step in the Decision layer, after customer segmentation with the RF model, is to develop a prediction model to estimate the CRP level. The previous classification model categorized customers into three strategic segments (Loyal, At-Risk, and Critical). However, this stage shifts the focus to assessing how likely a customer is to remain engaged in repeat interactions with the company. The LR model uses output from LLMs, focusing on linguistic features such as sentiment polarity, engagement level, and intent score, all of which were generated earlier in the Processing and Integration layers. Figure 5 shows the ROC curve of LR for CRP. The ROC curve of the LR model is consistently above the diagonal line, with an AUC value of 0.84. The further the ROC curve is from the diagonal line representing the random classifier, the better the model's ability to distinguish between retained and churned customers. This result is in line with the findings in, which show that LR tends to produce AUC values of around 0.83–0.85, while the RF and Gradient Boosting (GB) models can achieve higher performance in detecting non-linear patterns of customer behavior. A similar study on telecommunication companies in the US reported that LR achieved an AUC of 0.84, while ensemble models such as RF and XGBoost improved to 0.93–0.96 on customer churn datasets. Consistent results were also found in, where LR was the most stable and well-interpreted basic model for predicting customer behavior in the banking sector, with an average AUC value of 0.80–0.88, although less accurate than the decision tree-based model. Furthermore, in, it was emphasized that the main advantage of LR is not in the complexity of the modeling, but in the clarity of the relationship between linguistic variables and customer behavior, which is important to support explainable business intelligence. Meanwhile, studies in emphasized that LR remains the benchmark model in customer churn prediction, due to its computationally efficient nature and ease of integration with natural language-based systems, such as LLMs, to support decision-making. Therefore, although non-linear models such as RF, XGBoost, and Neural Networks (NNs) have higher performance, LR still offers consistent performance, is easy to interpret, and is computationally efficient, making it very relevant to be implemented in the Decision layer in LLM-based CVM systems, especially when business decisions demand transparency, traceability, and clarity of cause-and-effect relationships between linguistic variables and customer behavior. IV. CONCLUSION. This study introduced an integrated framework that combines LLMs with CVM to analyze real-time customer conversations in a B2B environment. The proposed system can process customer interaction data from various communication channels, extracting linguistic and emotional meaning, and mapping the results into customer value indicators, such as CSI, CLV, and CRP. The test results indicate that integrating LLMs into B2B conversation analysis can enhance the accuracy and depth of understanding of customer communication context. The RF classifier achieved 91% accuracy in segmenting customers into three strategic groups (Loyal, At-Risk, and Critical), while the LR model achieved an AUC of 0.83 in predicting CRP. Both models show a balance between performance and interpretability, with RF excelling in non-linear classification and LR offering stable and easy-to-interpret results. The novelty of this work lies in its multi-layer framework that connects LLM-based sentiment, intent, and engagement signals to quantitative CVM indicators—an area absent in existing research. Unlike previous studies that focused on static sentiment analysis, this study offers a real-time, dashboard-driven approach tailored to Indonesian B2B e-commerce data. This contribution provides a scalable foundation for explainable data-driven customer value management. In further research, it is necessary to explore more adaptive and computationally efficient LLMs, while also integrating reinforcement learning techniques or dynamic knowledge graphs to enhance the system's ability to learn continuously and operate autonomously in context decision-making. Thus, this framework can be developed into an intelligent analytics platform that is not only reactive but also proactive in generating future customer value strategies. DATA AVAILABILITY STATEMENT The dataset used in this study consists of proprietary B2B customer conversations obtained from a restricted-access e-commerce platform. Due to contractual confidentiality and data privacy regulations, the dataset cannot be publicly shared. Access may be granted upon reasonable request and with the permission of the platform owner.