Predicting user satisfaction from customer service chats
Customer service is a determining factor in the user experience of Fintech companies. This work seeks to understand, using machine learning techniques, what factors lead the clients of a specific Fintech company to positively evaluate their experience. Two data sources were used to achieve this: us...
| Autores principales: | , |
|---|---|
| Formato: | Articulo |
| Lenguaje: | Español |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | http://sedici.unlp.edu.ar/handle/10915/168749 |
| Aporte de: |
| id |
I19-R120-10915-168749 |
|---|---|
| record_format |
dspace |
| spelling |
I19-R120-10915-1687492024-08-20T20:04:05Z http://sedici.unlp.edu.ar/handle/10915/168749 Predicting user satisfaction from customer service chats Predicción de la satisfacción del usuario a partir de chats de atención al cliente Romanisio, Alejandro Gravano, Agustín 2024-04 2024-08-20T17:41:03Z es Ciencias Informáticas customer service satisfaction surveys predictive models XGBoost natural language processing atención al cliente encuestas de satisfacción modelos predictivos procesamiento de lenguaje natural Customer service is a determining factor in the user experience of Fintech companies. This work seeks to understand, using machine learning techniques, what factors lead the clients of a specific Fintech company to positively evaluate their experience. Two data sources were used to achieve this: user records from their sign up and the log of conversations with customer service via WhatsApp. We experimented with predictive models based on XGBoost, trained with features of the user context, the characteristics of the conversations and the semantics of the words used in the conversations. The results were lower than expected (AUC = 0.5152), but they leave valuable lessons for those who face similar problems in the future, related to the challenges of the following critical aspects: i. avoid data leakage, ii. evaluate models and scoring metrics thoroughly, iii. carry out intermediate checkpoints, iv. do not underestimate the time required for data transformation, v. perform a unit testing process and vi. know the domain. This paper describes the different stages of the methodology: data extraction and transformation, feature generation, predictive model training, optimal model selection and test data evaluation. Los servicios de atención al cliente son determinantes de la experiencia de usuario de las empresas Fintech. Este trabajo busca entender, empleando técnicas de machine learning, qué factores llevan a los clientes de una Fintech a evaluar de forma positiva su experiencia. Esto se hizo a partir de dos fuentes de datos: los registros de los usuarios y las conversaciones del servicio de atención al cliente vía WhatsApp. Experimentamos con modelos predictivos basados en XGBoost, entrenados con features del contexto del usuario, las características de las conversaciones y la semántica de las palabras utilizadas en las conversaciones. Los resultados fueron menores a lo esperado (AUC = 0.5152), pero dejan aprendizajes valiosos para quienes encaren problemas semejantes en el futuro, relacionados a los desafíos de los siguientes aspectos críticos: i. evitar el data leakage, ii. evaluar modelos y scoring metrics exhaustivamente, iii. realizar chequeos intermedios, iv. no subestimar el tiempo necesario para la transformación de datos, v. realizar un proceso de unit testing y vi. conocer el dominio. Este trabajo describe las distintas etapas de la metodología: extracción y transformación de los datos, generación de features, entrenamiento de modelos predictivos, selección del modelo óptimo y evaluación en datos de test. Sociedad Argentina de Informática e Investigación Operativa Articulo Articulo http://creativecommons.org/licenses/by-nc/4.0/ Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) application/pdf 2-24 |
| institution |
Universidad Nacional de La Plata |
| institution_str |
I-19 |
| repository_str |
R-120 |
| collection |
SEDICI (UNLP) |
| language |
Español |
| topic |
Ciencias Informáticas customer service satisfaction surveys predictive models XGBoost natural language processing atención al cliente encuestas de satisfacción modelos predictivos procesamiento de lenguaje natural |
| spellingShingle |
Ciencias Informáticas customer service satisfaction surveys predictive models XGBoost natural language processing atención al cliente encuestas de satisfacción modelos predictivos procesamiento de lenguaje natural Romanisio, Alejandro Gravano, Agustín Predicting user satisfaction from customer service chats |
| topic_facet |
Ciencias Informáticas customer service satisfaction surveys predictive models XGBoost natural language processing atención al cliente encuestas de satisfacción modelos predictivos procesamiento de lenguaje natural |
| description |
Customer service is a determining factor in the user experience of Fintech companies. This work seeks to understand, using machine learning techniques, what factors lead the clients of a specific Fintech company to positively evaluate their experience. Two data sources were used to achieve this:
user records from their sign up and the log of conversations with customer service via WhatsApp. We experimented with predictive models based on XGBoost, trained with features of the user context, the characteristics of the conversations and the semantics of the words used in the conversations. The results were lower than expected (AUC = 0.5152), but they leave valuable lessons for those who face similar problems in the future, related to the challenges of the following critical aspects: i. avoid data leakage, ii. evaluate models and scoring metrics thoroughly, iii. carry out intermediate checkpoints, iv. do not underestimate the time required for data transformation, v. perform a unit testing process and vi. know the domain. This paper describes the different stages of the methodology: data extraction and transformation, feature generation, predictive model training, optimal model selection and test data evaluation. |
| format |
Articulo Articulo |
| author |
Romanisio, Alejandro Gravano, Agustín |
| author_facet |
Romanisio, Alejandro Gravano, Agustín |
| author_sort |
Romanisio, Alejandro |
| title |
Predicting user satisfaction from customer service chats |
| title_short |
Predicting user satisfaction from customer service chats |
| title_full |
Predicting user satisfaction from customer service chats |
| title_fullStr |
Predicting user satisfaction from customer service chats |
| title_full_unstemmed |
Predicting user satisfaction from customer service chats |
| title_sort |
predicting user satisfaction from customer service chats |
| publishDate |
2024 |
| url |
http://sedici.unlp.edu.ar/handle/10915/168749 |
| work_keys_str_mv |
AT romanisioalejandro predictingusersatisfactionfromcustomerservicechats AT gravanoagustin predictingusersatisfactionfromcustomerservicechats AT romanisioalejandro predicciondelasatisfacciondelusuarioapartirdechatsdeatencionalcliente AT gravanoagustin predicciondelasatisfacciondelusuarioapartirdechatsdeatencionalcliente |
| _version_ |
1809234808690180096 |