Performance Analysis on Parallel Data Loading based on Concurrency Features

Data migration and batch processing remain rudimentary processes in database systems while dealing with enormous volumes of data from multiple sources. Even before running Extract, Transform, Load (ETL) on parallel architectures, extensive querying and performance in a way challenges are required. R...

Full description

Saved in:
Bibliographic Details
Published in2022 International Conference on Decision Aid Sciences and Applications (DASA) pp. 1349 - 1354
Main Authors Rahman, Mohammad Ashekur, Hussna, Asma Ul, George, Fabian Parsia, Latif, Mir Lubna, Mehrin, Yousra, Esfar-E-Alam, A.M.
Format Conference Proceeding
LanguageEnglish
Published IEEE 23.03.2022
Subjects
Batch production systems
concurrency
Concurrent computing
data loading
data warehouse
ETL
Loading
Oracle
parallel multi-processing
Parallel processing
Performance gain
SQLite
Transforms
Writing
Online AccessGet full text

Cover

More Information
Summary:Data migration and batch processing remain rudimentary processes in database systems while dealing with enormous volumes of data from multiple sources. Even before running Extract, Transform, Load (ETL) on parallel architectures, extensive querying and performance in a way challenges are required. Real-time data analysis, together with data aggregation and transformation, remains a problem for decision-making since data warehouses retain historical data and update it on a regular basis. However, optimization lessens resource consumption as well as ensures parallel processing efficiency while reducing the time window. The ultimate purpose of this paper is to instantly improve the performance of parallel data loading through concurrency. The conducted analysis shows concurrency on the Oracle database significantly improves the performance gain along with data loading time.
DOI:10.1109/DASA54658.2022.9765048