Customer churn prediction in telecom using machine learning in big data platform

• Published in: Journal of big data Vol. 6; no. 1; pp. 1 - 24
• Main Authors: Ahmad, Abdelrahim Kasem, Jafar, Assef, Aljoumaa, Kadan
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 20.03.2019; Springer Nature B.V; SpringerOpen
• Subjects: Algorithms; Artificial intelligence; Big Data; Churn in telecom; Classification; Communications Engineering; Computational Science and Engineering; Computer Science; Customer churn prediction; Customers; Data management; Data Mining and Knowledge Discovery; Database Management; Decision trees; Feature extraction; Feature selection; Information Storage and Retrieval; Machine learning; Mathematical Applications in Computer Science; Mathematical models; Mobile Social Network Analysis; Network analysis; Networks; Performance evaluation; Prediction models; Social networks; Telecommunications; Telecommunications industry; Customer churn prediction; Feature selection; Machine learning; Classification; Mobile Social Network Analysis; Big data; Churn in telecom
• ISSN: 2196-1115; 2196-1115
• DOI: 10.1186/s40537-019-0191-6

Customer churn is a major problem and one of the most important concerns for large companies. Due to the direct effect on the revenues of the companies, especially in the telecom field, companies are seeking to develop means to predict potential customer to churn. Therefore, finding factors that increase customer churn is important to take necessary actions to reduce this churn. The main contribution of our work is to develop a churn prediction model which assists telecom operators to predict customers who are most likely subject to churn. The model developed in this work uses machine learning techniques on big data platform and builds a new way of features’ engineering and selection. In order to measure the performance of the model, the Area Under Curve (AUC) standard measure is adopted, and the AUC value obtained is 93.3%. Another main contribution is to use customer social network in the prediction model by extracting Social Network Analysis (SNA) features. The use of SNA enhanced the performance of the model from 84 to 93.3% against AUC standard. The model was prepared and tested through Spark environment by working on a large dataset created by transforming big raw data provided by SyriaTel telecom company. The dataset contained all customers’ information over 9 months, and was used to train, test, and evaluate the system at SyriaTel. The model experimented four algorithms: Decision Tree, Random Forest, Gradient Boosted Machine Tree “GBM” and Extreme Gradient Boosting “XGBOOST”. However, the best results were obtained by applying XGBOOST algorithm. This algorithm was used for classification in this churn predictive model.