MHBase: A Distributed Real-Time Query Scheme for Meteorological Data Based on HBase
Meteorological technology has evolved rapidly in recent years to provide enormous, accurate and personalized advantages in the public service. Large volumes of observational data are generated gradually by technologies such as geographical remote sensing, meteorological radar satellite, etc. that ma...
Saved in:
Published in | Future internet Vol. 8; no. 1; p. 6 |
---|---|
Main Authors | , , , , |
Format | Journal Article |
Language | English |
Published |
MDPI AG
01.03.2016
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Meteorological technology has evolved rapidly in recent years to provide enormous, accurate and personalized advantages in the public service. Large volumes of observational data are generated gradually by technologies such as geographical remote sensing, meteorological radar satellite, etc. that makes data analysis in weather forecasting more precise but also poses a threat to the traditional method of data storage. In this paper, we present MHBase, (Meteorological data based on HBase (Hadoop Database), a distributed real-time query scheme for meteorological data based on HBase. The calibrated data obtained from terminal devices will be partitioned into HBase and persisted to HDFS (the Hadoop Distributed File System). We propose two algorithms (the Indexed Store and the Indexed Retrieve Algorithms) to implement a secondary index using HBase Coprocessors, which allow MHbase to provide high performance data querying on columns other than rowkey. Experimental results show that the performance of MHBase can satisfy the basic demands of meteorological business services. |
---|---|
AbstractList | Meteorological technology has evolved rapidly in recent years to provide enormous, accurate and personalized advantages in the public service. Large volumes of observational data are generated gradually by technologies such as geographical remote sensing, meteorological radar satellite, etc. that makes data analysis in weather forecasting more precise but also poses a threat to the traditional method of data storage. In this paper, we present MHBase, (Meteorological data based on HBase (Hadoop Database), a distributed real-time query scheme for meteorological data based on HBase. The calibrated data obtained from terminal devices will be partitioned into HBase and persisted to HDFS (the Hadoop Distributed File System). We propose two algorithms (the Indexed Store and the Indexed Retrieve Algorithms) to implement a secondary index using HBase Coprocessors, which allow MHbase to provide high performance data querying on columns other than rowkey. Experimental results show that the performance of MHBase can satisfy the basic demands of meteorological business services. |
Author | Jin, Yuanfeng Xu, Xichao Shen, Wenhai Ma, Tinghuai Tang, Meili |
Author_xml | – sequence: 1 givenname: Tinghuai surname: Ma fullname: Ma, Tinghuai – sequence: 2 givenname: Xichao surname: Xu fullname: Xu, Xichao – sequence: 3 givenname: Meili surname: Tang fullname: Tang, Meili – sequence: 4 givenname: Yuanfeng surname: Jin fullname: Jin, Yuanfeng – sequence: 5 givenname: Wenhai surname: Shen fullname: Shen, Wenhai |
BookMark | eNpN0E1PAjEQBuDGYCIiB_9Brx5WW2a_6g0BhQRiFDxvpt0plizUdJcD_94FDPE0H5l5Du8t6-z8jhi7l-IRQIkn63IhhRDpFetKpVSUKAGdf_0N69f1pr0QoAZpmnXZcjF9wZqe-ZCPXd0Ep_cNlfyTsIpWbkv8Y0_hwJfmm9rB-sAX1JAPvvJrZ7DiY2yQH4mS-x0_YXfs2mJVU_-v9tjX62Q1mkbz97fZaDiPDIBoIogJoEwNKCUs6YyMwFiW0mAyMEKTTTEGK01OmSUFJeoklnmexoJEghqgx2Znt_S4KX6C22I4FB5dcVr4sC4wNM5UVOQSU2ifNJYQ51mMBg0kmdE6yQaostZ6OFsm-LoOZC-eFMUx3OISLvwCETds2g |
CitedBy_id | crossref_primary_10_1016_j_procs_2018_03_071 crossref_primary_10_1177_1550147720908170 crossref_primary_10_3390_su15032442 crossref_primary_10_1080_2150704X_2018_1530480 crossref_primary_10_4018_JDM_2018070101 crossref_primary_10_1080_2150704X_2018_1508907 |
Cites_doi | 10.3923/itj.2010.116.123 10.1007/s12145-014-0179-x 10.1007/s12145-013-0126-2 10.4156/jcit.vol6.issue4.11 10.1016/j.is.2014.09.005 10.1145/1978915.1978919 10.2495/ICIE130812 10.1002/cpe.3222 10.1175/JCLI-D-11-00015.1 10.1147/JRD.2013.2240171 |
ContentType | Journal Article |
DBID | AAYXX CITATION DOA |
DOI | 10.3390/fi8010006 |
DatabaseName | CrossRef DOAJ Directory of Open Access Journals |
DatabaseTitle | CrossRef |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: DOA name: Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Engineering |
EISSN | 1999-5903 |
ExternalDocumentID | oai_doaj_org_article_81a635abbad34874acac357cbb572a97 10_3390_fi8010006 |
GroupedDBID | -DT .4I 3V. 5VS 7WY 8FE 8FG 8FL AADQD AAFWJ AAKPC AAYXX ABDBF ABUWG ACIHN ADBBV AEAQA AFKRA AFPKN AFZYC ALMA_UNASSIGNED_HOLDINGS ARAPS AZQEC BCNDV BENPR BEZIV BGLVJ BPHCQ CCPQU CITATION DWQXO E3Z EAP EBS EJD ESX FRNLG GNUQQ GROUPED_ABI_INFORM_COMPLETE GROUPED_DOAJ HCIFZ IPNFZ K60 K6V K6~ K7- KQ8 M0C M0N MODMG M~E OK1 P62 PIMPY PQBIZ PQBZA PQQKQ PROAC RIG RNS TR2 |
ID | FETCH-LOGICAL-c330t-34e33d6c3990feb7ec0a41d1ca52c0bef6a43f1c8e7fe93dab54188640e05ab33 |
IEDL.DBID | DOA |
ISSN | 1999-5903 |
IngestDate | Thu Jul 04 21:11:16 EDT 2024 Fri Aug 23 04:09:38 EDT 2024 |
IsDoiOpenAccess | true |
IsOpenAccess | true |
IsPeerReviewed | true |
IsScholarly | true |
Issue | 1 |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-c330t-34e33d6c3990feb7ec0a41d1ca52c0bef6a43f1c8e7fe93dab54188640e05ab33 |
OpenAccessLink | https://doaj.org/article/81a635abbad34874acac357cbb572a97 |
ParticipantIDs | doaj_primary_oai_doaj_org_article_81a635abbad34874acac357cbb572a97 crossref_primary_10_3390_fi8010006 |
PublicationCentury | 2000 |
PublicationDate | 2016-03-01 |
PublicationDateYYYYMMDD | 2016-03-01 |
PublicationDate_xml | – month: 03 year: 2016 text: 2016-03-01 day: 01 |
PublicationDecade | 2010 |
PublicationTitle | Future internet |
PublicationYear | 2016 |
Publisher | MDPI AG |
Publisher_xml | – name: MDPI AG |
References | ref13 ref12 Li (ref7) 2011; 6 ref15 ref14 ref30 ref11 ref10 Gantz (ref2) 2011 ref1 ref17 ref19 ref18 Özsu (ref27) 2011 ref24 ref23 ref26 ref25 ref20 George (ref6) 2011 Aravinth (ref9) 2015; 1 ref22 ref21 ref28 ref29 ref8 Xie (ref16) 2014; 49 ref4 ref3 ref5 |
References_xml | – ident: ref14 doi: 10.3923/itj.2010.116.123 – ident: ref1 – ident: ref3 – volume: 1 start-page: 252 year: 2015 ident: ref9 article-title: An efficient HADOOP frameworks SQOOP and ambari for big data processing publication-title: IJIRST contributor: fullname: Aravinth – ident: ref5 – ident: ref17 doi: 10.1007/s12145-014-0179-x – ident: ref8 doi: 10.1007/s12145-013-0126-2 – ident: ref20 – ident: ref29 – ident: ref24 – ident: ref22 – ident: ref25 – volume: 6 start-page: 92 year: 2011 ident: ref7 article-title: High Performance Computing Model for Processing Meteorological Data in Cluster System publication-title: J. Convergence Inf. Technol. doi: 10.4156/jcit.vol6.issue4.11 contributor: fullname: Li – ident: ref19 – ident: ref11 – ident: ref15 – ident: ref26 doi: 10.1016/j.is.2014.09.005 – ident: ref30 – ident: ref13 – ident: ref12 doi: 10.1145/1978915.1978919 – volume: 49 start-page: 691 year: 2014 ident: ref16 article-title: On massive spatial data cloud storage and quad-tree index based on the Hbase publication-title: WIT. Trans. Inf. Commun. Technol. doi: 10.2495/ICIE130812 contributor: fullname: Xie – start-page: 9 year: 2011 ident: ref2 article-title: Extracting value from chaos publication-title: IDC iview contributor: fullname: Gantz – ident: ref28 – ident: ref21 – ident: ref23 – start-page: 328 year: 2011 ident: ref6 article-title: The region life cycle contributor: fullname: George – ident: ref4 doi: 10.1002/cpe.3222 – ident: ref10 doi: 10.1175/JCLI-D-11-00015.1 – start-page: 497 year: 2011 ident: ref27 article-title: Parallel Database Systems contributor: fullname: Özsu – ident: ref18 doi: 10.1147/JRD.2013.2240171 |
SSID | ssj0000392667 |
Score | 2.0834868 |
Snippet | Meteorological technology has evolved rapidly in recent years to provide enormous, accurate and personalized advantages in the public service. Large volumes of... |
SourceID | doaj crossref |
SourceType | Open Website Aggregation Database |
StartPage | 6 |
SubjectTerms | distributed storage HBase meteorological data secondary index |
Title | MHBase: A Distributed Real-Time Query Scheme for Meteorological Data Based on HBase |
URI | https://doaj.org/article/81a635abbad34874acac357cbb572a97 |
Volume | 8 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV09a8MwEBUlXdqh9JN-BlG6iliRJSvd4qYhFFJo00A2o48TdHFKcIb--57sJLhTl44W5rDemXvvQHpHyAMIhaUXJHPI5iwFkbGBC5yB5wG4kuBqM53pq5rM05eFXLRGfcUzYY09cANcT3ODnGisNV6guE6NM07IzFkrs74ZNPfIuWw1U3UNRtpXKmushAT29b3wibU4FudfBNTy6a8JZXxMjjZKkA6bLzghe1CeksOWP-AZmU0nObLMIx3SUTS4jbOpwNN3FHcs3t2gb2tYfdMZAo8PKD_pFDXwcrUtaHRkKkNjCE-XJa2DnZP5-PnjacI2UxCYEyKpmEAAhVcOlUQSwGbgEpNyz52RfZdYCMqkInCnIQswEN5YmXKtVZpAgrgJcUE65bKES0I1akMenNfao5LLnPEJAhk1mvF97eUVud9CU3w1ZhcFNgkRv2KH3xXJI2i7F6I_db2AWSs2WSv-ytr1fwS5IQe4H9WcCLslnWq1hjuUCJXtkv1hPsrH3fqv-AHgYbvw |
link.rule.ids | 315,786,790,870,2115,27955,27956 |
linkProvider | Directory of Open Access Journals |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=MHBase%3A+A+Distributed+Real-Time+Query+Scheme+for+Meteorological+Data+Based+on+HBase&rft.jtitle=Future+internet&rft.au=Tinghuai+Ma&rft.au=Xichao+Xu&rft.au=Meili+Tang&rft.au=Yuanfeng+Jin&rft.date=2016-03-01&rft.pub=MDPI+AG&rft.eissn=1999-5903&rft.volume=8&rft.issue=1&rft.spage=6&rft_id=info:doi/10.3390%2Ffi8010006&rft.externalDBID=DOA&rft.externalDocID=oai_doaj_org_article_81a635abbad34874acac357cbb572a97 |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1999-5903&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1999-5903&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1999-5903&client=summon |