Large scale data storage and processing of insulator leakage current using HBase and mapreduce

In the smart grid environment, huge volumes of data will be accumulated from the condition monitoring system of power equipment. Using the traditional centralized storage architecture and relational databases, the performance of data querying and processing is slow, and cannot meet real-time require...

Full description

Saved in:
Bibliographic Details
Published in2014 International Conference on Power System Technology pp. 1331 - 1337
Main Authors Yaqi Song, Yongli Zhu, Li Li
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.10.2014
Subjects
cloud computing
Contamination
Distributed databases
Hadoop
HBase
insulator leakage current
Insulators
Leakage currents
MapReduce
Memory
Power systems
Training
Online AccessGet full text

Cover

More Information
Summary:In the smart grid environment, huge volumes of data will be accumulated from the condition monitoring system of power equipment. Using the traditional centralized storage architecture and relational databases, the performance of data querying and processing is slow, and cannot meet real-time requirements of the power equipment condition monitoring system. Meanwhile, MapReduce is a desirable parallel programming platform that is widely applied in kinds of data process fields. In this paper, a case-study on distributed storage using HBase and parallel processing of insulator leakage current data is presented. We propose efficient MapReduce based algorithms for parallel join query, parallel characteristics extraction and analogous assessment of insulator contamination degree. We evaluate our work on real large scale datasets utilizing Hadoop platform. Results reveal that the speedup and scale-up of our work are competent.
DOI:10.1109/POWERCON.2014.6993650