A Hybrid MapReduce Implementation of PCA on Tianhe-2

"Big Data" has been a popular word anywhere. Researchers want the data processing more efficient. PCA algorithm is an effective data reduction algorithm applied to almost all big data fields. Meanwhile, there are many Machine Learning Algorithm Library applied to provide commonly-used algo...

Full description

Saved in:
Bibliographic Details
Published inJournal of physics. Conference series Vol. 1168; no. 5; pp. 52013 - 52019
Main Authors Yu, Wei, Qu, Yili, Lu, Yutong
Format Journal Article
LanguageEnglish
Published Bristol IOP Publishing 01.02.2019
Subjects
Algorithms
Big Data
Data processing
Data reduction
Machine learning
Online AccessGet full text

Cover

Abstract "Big Data" has been a popular word anywhere. Researchers want the data processing more efficient. PCA algorithm is an effective data reduction algorithm applied to almost all big data fields. Meanwhile, there are many Machine Learning Algorithm Library applied to provide commonly-used algorithm, but these algorithms do not make good use of the resources of the supercomputer system. This paper uses MapReduce Model to design and implement PCA algorithm using MPI + OpenMP + SIMD hybrid accelerator programming tools on Tianhe-2 and get a significant speedup.
AbstractList "Big Data" has been a popular word anywhere. Researchers want the data processing more efficient. PCA algorithm is an effective data reduction algorithm applied to almost all big data fields. Meanwhile, there are many Machine Learning Algorithm Library applied to provide commonly-used algorithm, but these algorithms do not make good use of the resources of the supercomputer system. This paper uses MapReduce Model to design and implement PCA algorithm using MPI + OpenMP + SIMD hybrid accelerator programming tools on Tianhe-2 and get a significant speedup.
Author Lu, Yutong
Yu, Wei
Qu, Yili
Author_xml – sequence: 1
  givenname: Wei
  surname: Yu
  fullname: Yu, Wei
  organization: College of Computer, National University of Defense Technology , China
– sequence: 2
  givenname: Yili
  surname: Qu
  fullname: Qu, Yili
  organization: School of Data and Computer Science, Sun Yat-Sen University , China
– sequence: 3
  givenname: Yutong
  surname: Lu
  fullname: Lu, Yutong
  email: ytlu@nudt.edu.cn
  organization: National Supercomputer Center in Guangzhou , China
BookMark eNqFUE1Lw0AQXaSCbfU3GPAcs7PfeyxFbaGiSD0vyWYXU9ps3CSH_nsTIvXoXOYN894b5i3QrA61Q-ge8CNgpTKQjKSCa5EBCJXxDHOCgV6h-WUzu2ClbtCibQ8Y06HkHLFVsjkXsSqT17z5cGVvXbI9NUd3cnWXd1Wok-CT9_UqGdC-yusvl5JbdO3zY-vufvsSfT4_7debdPf2sl2vdqmlhHWpACaBceullUIorwvGLLWYUOmZ1VRroiwVwKGUijBMh0Fq6QsHOFda0yV6mHybGL5713bmEPpYDycN4YITpUGTgSUnlo2hbaPzponVKY9nA9iMEZnxeTMGYcaIDDdTRIOSTsoqNH_W_6l-AM16ZgI
Cites_doi 10.1007/s10619-013-7134-6
10.1007/BF02940959
10.1145/1327452.1327492
10.1037/h0071325
ContentType Journal Article
Copyright Published under licence by IOP Publishing Ltd
2019. This work is published under http://creativecommons.org/licenses/by/3.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: Published under licence by IOP Publishing Ltd
– notice: 2019. This work is published under http://creativecommons.org/licenses/by/3.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID O3W
TSCCA
AAYXX
CITATION
8FD
8FE
8FG
ABUWG
AFKRA
ARAPS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
H8D
HCIFZ
L7M
P5Z
P62
PIMPY
PQEST
PQQKQ
PQUKI
PRINS
DOI 10.1088/1742-6596/1168/5/052013
DatabaseName IOP Publishing
IOPscience (Open Access)
CrossRef
Technology Research Database
ProQuest SciTech Collection
ProQuest Technology Collection
ProQuest Central (Alumni)
ProQuest Central
Advanced Technologies & Aerospace Database‎ (1962 - current)
ProQuest Central Essentials
ProQuest Central
Technology Collection
ProQuest One Community College
ProQuest Central
Aerospace Database
SciTech Premium Collection (Proquest) (PQ_SDU_P3)
Advanced Technologies Database with Aerospace
ProQuest Advanced Technologies & Aerospace Database
ProQuest Advanced Technologies & Aerospace Collection
Publicly Available Content Database
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Academic
ProQuest One Academic UKI Edition
ProQuest Central China
DatabaseTitle CrossRef
Publicly Available Content Database
Advanced Technologies & Aerospace Collection
Technology Collection
Technology Research Database
ProQuest Advanced Technologies & Aerospace Collection
ProQuest Central Essentials
ProQuest One Academic Eastern Edition
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Technology Collection
ProQuest SciTech Collection
ProQuest Central China
ProQuest Central
Advanced Technologies & Aerospace Database
Aerospace Database
ProQuest One Academic UKI Edition
ProQuest Central Korea
ProQuest One Academic
Advanced Technologies Database with Aerospace
DatabaseTitleList
Publicly Available Content Database
Database_xml – sequence: 1
  dbid: O3W
  name: IOP Publishing
  url: http://iopscience.iop.org/
  sourceTypes: Publisher
– sequence: 2
  dbid: 8FG
  name: ProQuest Technology Collection
  url: https://search.proquest.com/technologycollection1
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Physics
DocumentTitleAlternate A Hybrid MapReduce Implementation of PCA on Tianhe-2
EISSN 1742-6596
ExternalDocumentID 10_1088_1742_6596_1168_5_052013
JPCS_1168_5_052013
GroupedDBID 1JI
29L
2WC
4.4
5B3
5GY
5PX
5VS
7.Q
AAJIO
AAJKP
ABHWH
ACAFW
ACHIP
AEFHF
AEJGL
AFKRA
AFYNE
AIYBF
AKPSB
ALMA_UNASSIGNED_HOLDINGS
ARAPS
ASPBG
ATQHT
AVWKF
AZFZN
BENPR
BGLVJ
CCPQU
CEBXE
CJUJL
CRLBU
CS3
DU5
E3Z
EBS
EDWGO
EJD
EQZZN
F5P
FRP
GROUPED_DOAJ
GX1
HCIFZ
HH5
IJHAN
IOP
IZVLO
J9A
KNG
KQ8
LAP
N5L
N9A
O3W
OK1
P2P
PIMPY
PJBAE
RIN
RNS
RO9
ROL
SY9
T37
TR2
TSCCA
UCJ
W28
XSB
~02
02O
1WK
AALHV
AAYXX
AERVB
AHSEE
BBWZM
C1A
CITATION
FEDTE
H13
HVGLF
JCGBZ
M48
Q02
S3P
8FD
8FE
8FG
ABUWG
AZQEC
DWQXO
H8D
L7M
P62
PQEST
PQQKQ
PQUKI
PRINS
ID FETCH-LOGICAL-c324t-6147145cf7c7668f9b44c3c0237f4c939928c36151d782403c36797fbe10a8993
IEDL.DBID IOP
ISSN 1742-6588
IngestDate Thu Oct 10 20:28:36 EDT 2024
Thu Sep 26 16:30:19 EDT 2024
Wed Aug 21 03:40:17 EDT 2024
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 5
Language English
License Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c324t-6147145cf7c7668f9b44c3c0237f4c939928c36151d782403c36797fbe10a8993
OpenAccessLink https://proxy.k.utb.cz/login?url=https://iopscience.iop.org/article/10.1088/1742-6596/1168/5/052013
PQID 2565289192
PQPubID 4998668
PageCount 7
ParticipantIDs crossref_primary_10_1088_1742_6596_1168_5_052013
iop_journals_10_1088_1742_6596_1168_5_052013
proquest_journals_2565289192
PublicationCentury 2000
PublicationDate 20190201
PublicationDateYYYYMMDD 2019-02-01
PublicationDate_xml – month: 02
  year: 2019
  text: 20190201
  day: 01
PublicationDecade 2010
PublicationPlace Bristol
PublicationPlace_xml – name: Bristol
PublicationTitle Journal of physics. Conference series
PublicationTitleAlternate J. Phys.: Conf. Ser
PublicationYear 2019
Publisher IOP Publishing
Publisher_xml – name: IOP Publishing
References 11
12
14
15
Zaharia M (5) 2010
Pearson K. (13) 1901; 2
1
2
3
4
6
7
8
9
Benson A R (16) 2013
10
References_xml – ident: 3
– ident: 4
– ident: 12
– ident: 11
– ident: 15
  doi: 10.1007/s10619-013-7134-6
– start-page: 264
  year: 2013
  ident: 16
  publication-title: IEEE International Conference on Big Data
  contributor:
    fullname: Benson A R
– ident: 2
  doi: 10.1007/BF02940959
– ident: 10
– volume: 2
  start-page: 559
  issn: 0031-8086
  year: 1901
  ident: 13
  publication-title: Philosophical Magazine
  contributor:
    fullname: Pearson K.
– start-page: 10
  year: 2010
  ident: 5
  publication-title: Usenix Conference on Hot Topics in Cloud Computing
  contributor:
    fullname: Zaharia M
– ident: 6
– ident: 9
– ident: 7
– ident: 8
– ident: 1
  doi: 10.1145/1327452.1327492
– ident: 14
  doi: 10.1037/h0071325
SSID ssj0033337
Score 2.2253933
Snippet "Big Data" has been a popular word anywhere. Researchers want the data processing more efficient. PCA algorithm is an effective data reduction algorithm...
“Big Data” has been a popular word anywhere. Researchers want the data processing more efficient. PCA algorithm is an effective data reduction algorithm...
SourceID proquest
crossref
iop
SourceType Aggregation Database
Publisher
StartPage 52013
SubjectTerms Algorithms
Big Data
Data processing
Data reduction
Machine learning
SummonAdditionalLinks – databaseName: ProQuest Technology Collection
  dbid: 8FG
  link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwhV3NS8MwFA86EbyInzidkoNHQ5smaZKTjOEcwkRkg91Kk6bowa66efC_96VNGSJoT_0-_F54v_de3gdC15o7ZpSwxBlREF7EkuS5AWdFWp045qhyPg45fUwnc_6wEIsQcFuFtMpOJzaKulhaHyOPgJoFOAdgkNzW78RPjfK7q2GExjbaoYmU3vlS4_tOEzM4ZFsQmRBgWtXld4HTF-7pNKI0VZGIfD4IZT_Yaft1Wf9S0Q3vjA_QfjAY8bCV8CHactUR2m0SN-3qGPEhnnz5qis8zetn34fV4abj71soKqrwssRPoyGGsxmshRdHkhM0H9_NRhMSJiEQCwbPGvw74BAubCmtTFNVasO5ZRb4Vpbcat9cVlnmjZMCGJ_HDC6klqVxNM7Bo2KnqFctK3eGsOGsLEWap9SUXPjaAwqkbROngemVSvso7hDI6rbhRdZsVCuVedAyD1rmQctE1oLWRzeAVBYW_-r_1wcdpJtvNuI9__vxBdqDn-g2cXqAeuuPT3cJdsHaXDXC_wYDuKtp
  priority: 102
  providerName: ProQuest
Title A Hybrid MapReduce Implementation of PCA on Tianhe-2
URI https://iopscience.iop.org/article/10.1088/1742-6596/1168/5/052013
https://www.proquest.com/docview/2565289192
Volume 1168
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1bS8MwFA5uQ_DFuzidpQ8-2rVdLk0f59icwi6MDfcWmixFELvhugf99Z70gjdExD60KfSE9GuS8530nBOELkOiseRUOVrShUMWXuBEkQRjJVBhS2Ptc23WIQdD1p-Ruzmdf4yFWa6Kqb8JxTxRcA5h4RDHXeDQLYfRkLm-z7hLXePKYTaurWEYMcYAux2Ny9kYwxHkQZFGiPPSx-vnij5pqAq04ts0neme3h5SZatzl5PH5iaVTfX6JaHj_15rH-0W1NRu5xIHaEsnh2g7cxFV6yNE2nb_xcR32YNoNTEZX7Wd5RZ-KsKXEnsZ2-NO24bSFHrdg3Zax2jW6047fafYc8FRQK1SsCRBWxGq4kAFjPE4lIQorECzBzFRoUljyxU2NGgB3IJ4GG6CMIil9r0IbDd8gqrJMtGnyJYExzFlEfNlTKiJcvCBHqiWDoFTcM7qyCtxFqs8tYbIfolzLgwYwoAhDBiCihyMOroC-EQxzNa_P94oP9y7DHA8ClYmMNuzv9V2jnbgEuYu2w1UTZ83-gIYSSotVOG9GwvVrrvD8cQy2oFaWTeE8wjfvwEH_dAT
link.rule.ids 315,783,787,12777,21400,27936,27937,33385,33756,38877,38902,43612,43817,53854,53880
linkProvider IOP Publishing
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwhV07T8MwELZoKwQL4ikKBTIwEuVhO3EmVKpWAdqqqlqpmxU7jmAgCbQM_HvOiaMKIUGmvIfPp_vuzvdA6DYiCgtGpa0ETW2SuqGdJAKclVBGvsLKY0rHISfTIF6SpxVdmYDb2qRVNjqxUtRpIXWM3AFqpuAcgEFyX77bemqU3l01IzRaqKNbVYHz1XkYTmfzRhdjOMK6JNK3gWtZk-EFbp-5FwWO5wXMoY7OCPHwD35qvRblLyVdMc_oEB0Yk9Hq12t8hHZUfox2q9RNuT5BpG_FX7ruypok5Vx3YlVW1fP3zZQV5VaRWbNB34KzBUjDi7L9U7QcDReD2DazEGwJJs8GPDxgEUJlFsowCFgWCUIklsC4YUZkpNvLMom1eZIC5xMXw0UYhZlQnpuAT4XPUDsvcnWOLEFwltEgCTyREaqrDzzATvoqAq5nLOgit0GAl3XLC15tVTPGNWhcg8Y1aJzyGrQuugOkuBH_9f-v9xpIt99sF_ji78c3aC9eTMZ8_Dh9vkT78MOoTqPuofbm41NdgZWwEddGFL4BRoCvug
linkToPdf http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3JTsMwELXaIhAXxCoKBXLgSMjiJc6xKkRlaalQK3qzYscWF9KIlgN_zzgLCCGEyMmJYst6msx7k8xMEDqPicaSU-VqSTOXZH7kpqmEYCVScaixDri27yFHYzackds5nbdQ8lkLsyhq138Jw6pRcAVhnRDHPdDQoctozLwgYNyjnk3lCLBXZKaN1qjtbgJ2_YCfGo-M4Yiqwkg7kfMmz-v3xb6xVBt28sNVl_yTbKOtWjg6_WqbO6il8120XiZwquUeIn1n-G6rr5xRWjzafqzaKTv_vtTFRbmzMM5k0HdgNAWbeNZuuI9myfV0MHTrPyK4CoTPCuI84BJClYlUxBg3sSREYQW8GxmiYttklitsRUoGzE98DCdRHBmpAz-FyAofoE6-yPUhciTBxlCWskAaQm0NQgDkrUIdA-NzzrrIbxAQRdX4QpQfrDkXFjRhQRMWNEFFBVoXXQBSon4Iln_f3msg_ZoDCoxCDAi68-h_q52hjclVIu5vxnfHaBOuxFVudQ91Vq9v-gSkw0qelnbxAQnxr5c
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Hybrid+MapReduce+Implementation+of+PCA+on+Tianhe-2&rft.jtitle=Journal+of+physics.+Conference+series&rft.au=Yu%2C+Wei&rft.au=Qu%2C+Yili&rft.au=Lu%2C+Yutong&rft.date=2019-02-01&rft.issn=1742-6588&rft.eissn=1742-6596&rft.volume=1168&rft.spage=52013&rft_id=info:doi/10.1088%2F1742-6596%2F1168%2F5%2F052013&rft.externalDBID=n%2Fa&rft.externalDocID=10_1088_1742_6596_1168_5_052013
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1742-6588&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1742-6588&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1742-6588&client=summon