CP decomposition for tensors via alternating least squares with QR decomposition
Abstract The CP tensor decomposition is used in applications such as machine learning and signal processing to discover latent low‐rank structure in multidimensional data. Computing a CP decomposition via an alternating least squares (ALS) method reduces the problem to several linear least squares p...
Saved in:
Published in | Numerical linear algebra with applications Vol. 30; no. 6 |
---|---|
Main Authors | , , , |
Format | Journal Article |
Language | English |
Published |
Oxford
Wiley Subscription Services, Inc
01.12.2023
Wiley |
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Abstract
The CP tensor decomposition is used in applications such as machine learning and signal processing to discover latent low‐rank structure in multidimensional data. Computing a CP decomposition via an alternating least squares (ALS) method reduces the problem to several linear least squares problems. The standard way to solve these linear least squares subproblems is to use the normal equations, which inherit special tensor structure that can be exploited for computational efficiency. However, the normal equations are sensitive to numerical ill‐conditioning, which can compromise the results of the decomposition. In this paper, we develop versions of the CP‐ALS algorithm using the QR decomposition and the singular value decomposition, which are more numerically stable than the normal equations, to solve the linear least squares problems. Our algorithms utilize the tensor structure of the CP‐ALS subproblems efficiently, have the same complexity as the standard CP‐ALS algorithm when the input is dense and the rank is small, and are shown via examples to produce more stable results when ill‐conditioning is present. Our MATLAB implementation achieves the same running time as the standard algorithm for small ranks, and we show that the new methods can obtain lower approximation error. |
---|---|
AbstractList | The CP tensor decomposition is used in applications such as machine learning and signal processing to discover latent low-rank structure in multidimensional data. Computing a CP decomposition via an alternating least squares (ALS) method reduces the problem to several linear least squares problems. The standard way to solve these linear least squares subproblems is to use the normal equations, which inherit special tensor structure that can be exploited for computational efficiency. However, the normal equations are sensitive to numerical ill-conditioning, which can compromise the results of the decomposition. In this paper, we develop versions of the CP-ALS algorithm using the QR decomposition and the singular value decomposition, which are more numerically stable than the normal equations, to solve the linear least squares problems. Our algorithms utilize the tensor structure of the CP-ALS subproblems efficiently, have the same complexity as the standard CP-ALS algorithm when the input is dense and the rank is small, and are shown via examples to produce more stable results when ill-conditioning is present. Our MATLAB implementation achieves the same running time as the standard algorithm for small ranks, and we show that the new methods can obtain lower approximation error. Abstract The CP tensor decomposition is used in applications such as machine learning and signal processing to discover latent low‐rank structure in multidimensional data. Computing a CP decomposition via an alternating least squares (ALS) method reduces the problem to several linear least squares problems. The standard way to solve these linear least squares subproblems is to use the normal equations, which inherit special tensor structure that can be exploited for computational efficiency. However, the normal equations are sensitive to numerical ill‐conditioning, which can compromise the results of the decomposition. In this paper, we develop versions of the CP‐ALS algorithm using the QR decomposition and the singular value decomposition, which are more numerically stable than the normal equations, to solve the linear least squares problems. Our algorithms utilize the tensor structure of the CP‐ALS subproblems efficiently, have the same complexity as the standard CP‐ALS algorithm when the input is dense and the rank is small, and are shown via examples to produce more stable results when ill‐conditioning is present. Our MATLAB implementation achieves the same running time as the standard algorithm for small ranks, and we show that the new methods can obtain lower approximation error. |
Author | Liu, Xiaotian Ballard, Grey Minster, Rachel Viviano, Irina |
Author_xml | – sequence: 1 givenname: Rachel surname: Minster fullname: Minster, Rachel organization: Department of Computer Science Wake Forest University Winston‐Salem North Carolina USA – sequence: 2 givenname: Irina surname: Viviano fullname: Viviano, Irina organization: Clinical & Translational Science Institute Wake Forest University School of Medicine Winston‐Salem North Carolina USA – sequence: 3 givenname: Xiaotian orcidid: 0000-0001-9369-4029 surname: Liu fullname: Liu, Xiaotian organization: Department of Computer Science Wake Forest University Winston‐Salem North Carolina USA – sequence: 4 givenname: Grey surname: Ballard fullname: Ballard, Grey organization: Department of Computer Science Wake Forest University Winston‐Salem North Carolina USA |
BackLink | https://www.osti.gov/servlets/purl/1987855$$D View this record in Osti.gov |
BookMark | eNpVkMtKQzEURYNUsK2CnxB04uTWPG5eQym-oGAVHYc0ybUpt0mbpIp_b0udODp7sPaGs0ZgEFP0AFxiNMEIkdvYmwlhGJ-AIUZKNZghPjhkgRpGCTsDo1JWCCHOFB2C-XQOnbdpvUkl1JAi7FKG1ceScoFfwUDTV5-jqSF-wt6bUmHZ7kz2BX6HuoSvb__75-C0M33xF393DD4e7t-nT83s5fF5ejdrLMW8NgtMWkVbYVuFO8GN8x3nRCGrOKMYS-MWyDvkBCGt9LZ1jnvDjZALLxm2jo7B1XE3lRp0saF6u7QpRm-rxkoKydgeuj5Cm5y2O1-qXqXd_pm-aCIlo6IViu-pmyNlcyol-05vclib_KMx0gepei9VH6TSX06Ua5E |
CitedBy_id | crossref_primary_10_1002_nla_2542 |
Cites_doi | 10.21236/AD0705509 10.1137/1.9781611971446 10.1109/TSP.2018.2887192 10.1109/IPDPS.2015.27 10.1137/20M1344561 10.1109/IPDPS.2017.86 10.56021/9781421407944 10.1109/IPDPS.2019.00023 10.1134/S0965542513120129 10.1038/s41586-022-05172-4 10.1145/3432185 10.1137/060676489 10.1137/120868323 10.1007/978-3-319-64203-1_47 10.1145/2688500.2688513 10.1145/3378445 10.1109/HPEC.2012.6408676 10.1109/TSP.2013.2269903 10.1109/ICPP.2016.19 10.1137/07070111X 10.1109/IPDPS.2016.113 10.1007/BF02165411 10.1137/1.9780898719574 10.1137/040604959 10.1016/0020-0190(79)90113-3 10.1109/TSP.2017.2777399 10.1137/1.9781611976137.1 |
ContentType | Journal Article |
Copyright | 2023. This article is published under http://creativecommons.org/licenses/by-nc/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
Copyright_xml | – notice: 2023. This article is published under http://creativecommons.org/licenses/by-nc/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
CorporateAuthor | Wake Forest Univ., Winston-Salem, NC (United States) |
CorporateAuthor_xml | – name: Wake Forest Univ., Winston-Salem, NC (United States) |
DBID | AAYXX CITATION 7SC 7TB 8FD FR3 JQ2 KR7 L7M L~C L~D OIOZB OTOTI |
DOI | 10.1002/nla.2511 |
DatabaseName | CrossRef Computer and Information Systems Abstracts Mechanical & Transportation Engineering Abstracts Technology Research Database Engineering Research Database ProQuest Computer Science Collection Civil Engineering Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional OSTI.GOV - Hybrid OSTI.GOV |
DatabaseTitle | CrossRef Civil Engineering Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Mechanical & Transportation Engineering Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
DatabaseTitleList | CrossRef Civil Engineering Abstracts |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Mathematics |
EISSN | 1099-1506 |
ExternalDocumentID | 1987855 10_1002_nla_2511 |
GroupedDBID | -~X .3N .4S .DC .GA .Y3 05W 0R~ 10A 123 1L6 1OB 1OC 1ZS 31~ 33P 3SF 3WU 4.4 50Y 50Z 51W 51X 52M 52N 52O 52P 52S 52T 52U 52W 52X 5VS 66C 702 7PT 8-0 8-1 8-3 8-4 8-5 8UM 930 A03 AAESR AAEVG AAHHS AANLZ AAONW AASGY AAXRX AAYXX AAZKR ABCQN ABCUV ABEFU ABEML ABIJN ABPVW ACAHQ ACBWZ ACCFJ ACCZN ACGFS ACPOU ACSCC ACXBN ACXQS ADBBV ADEOM ADIZJ ADKYN ADMGS ADOZA ADXAS ADZMN AEEZP AEIGN AEIMD AENEX AEQDE AEUQT AEUYR AFBPY AFFPM AFGKR AFPWT AFZJQ AHBTC AITYG AIURR AIWBW AJBDE AJXKR ALAGY ALMA_UNASSIGNED_HOLDINGS ALUQN AMBMR AMYDB ARCSS ASPBG ATUGU AUFTA AVWKF AZBYB AZFZN AZVAB BAFTC BDRZF BFHJK BHBCM BMNLL BMXJE BNHUX BROTX BRXPI BY8 CITATION CS3 D-E D-F DCZOG DPXWK DR2 DRFUL DRSTM DU5 EBS EDO EJD F00 F01 F04 FEDTE G-S G.N GBZZK GNP GODZA H.T H.X HBH HF~ HGLYW HHY HVGLF HZ~ IX1 J0M JPC KQQ LATKE LAW LC2 LC3 LEEKS LH4 LITHE LOXES LP6 LP7 LUTES LW6 LYRES M6O MEWTI MK4 MRFUL MRSTM MSFUL MSSTM MXFUL MXSTM N04 N05 N9A NF~ O66 O9- OIG P2P P2W P2X P4D PALCI PQQKQ Q.N Q11 QB0 QRW R.K RIWAO RJQFR ROL RWI RWS RX1 RYL SAMSI SUPJJ TUS UB1 V2E W8V W99 WBKPD WIB WIH WIK WOHZO WQJ WRC WXSBR WYISQ XBAML XG1 XPP XV2 ZZTAW ~IA ~WT 7SC 7TB 8FD FR3 JQ2 KR7 L7M L~C L~D ABHUG ACXME ADAWD ADDAD AFVGU AGJLS OIOZB OTOTI |
ID | FETCH-LOGICAL-c316t-b1249347c491f76adef66290c9653118adb0ed0d72248ec4dd6ea6a78be851cd3 |
ISSN | 1070-5325 |
IngestDate | Mon Apr 01 04:54:49 EDT 2024 Fri Sep 13 03:16:08 EDT 2024 Fri Aug 23 00:51:53 EDT 2024 |
IsDoiOpenAccess | true |
IsOpenAccess | true |
IsPeerReviewed | true |
IsScholarly | true |
Issue | 6 |
Language | English |
LinkModel | OpenURL |
MergedId | FETCHMERGED-LOGICAL-c316t-b1249347c491f76adef66290c9653118adb0ed0d72248ec4dd6ea6a78be851cd3 |
Notes | SC0023296 USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) National Science Foundation (NSF) |
ORCID | 0000-0001-9369-4029 0000000193694029 |
OpenAccessLink | https://www.osti.gov/servlets/purl/1987855 |
PQID | 2885374796 |
PQPubID | 2034341 |
ParticipantIDs | osti_scitechconnect_1987855 proquest_journals_2885374796 crossref_primary_10_1002_nla_2511 |
PublicationCentury | 2000 |
PublicationDate | 2023-12-01 |
PublicationDateYYYYMMDD | 2023-12-01 |
PublicationDate_xml | – month: 12 year: 2023 text: 2023-12-01 day: 01 |
PublicationDecade | 2020 |
PublicationPlace | Oxford |
PublicationPlace_xml | – name: Oxford – name: United States |
PublicationTitle | Numerical linear algebra with applications |
PublicationYear | 2023 |
Publisher | Wiley Subscription Services, Inc Wiley |
Publisher_xml | – name: Wiley Subscription Services, Inc – name: Wiley |
References | e_1_2_8_28_1 e_1_2_8_29_1 e_1_2_8_24_1 e_1_2_8_25_1 e_1_2_8_26_1 e_1_2_8_27_1 e_1_2_8_3_1 e_1_2_8_2_1 e_1_2_8_5_1 e_1_2_8_7_1 e_1_2_8_6_1 e_1_2_8_9_1 e_1_2_8_8_1 Khatri C (e_1_2_8_17_1) 1968; 30 e_1_2_8_21_1 e_1_2_8_22_1 e_1_2_8_23_1 e_1_2_8_18_1 e_1_2_8_19_1 e_1_2_8_13_1 e_1_2_8_14_1 Kossaifi J (e_1_2_8_4_1) 2019; 20 e_1_2_8_15_1 e_1_2_8_16_1 Vervliet N (e_1_2_8_20_1) 2019 e_1_2_8_32_1 e_1_2_8_10_1 e_1_2_8_31_1 e_1_2_8_11_1 e_1_2_8_34_1 e_1_2_8_12_1 e_1_2_8_33_1 e_1_2_8_30_1 |
References_xml | – ident: e_1_2_8_29_1 doi: 10.21236/AD0705509 – ident: e_1_2_8_13_1 doi: 10.1137/1.9781611971446 – volume: 30 start-page: 167 issue: 2 year: 1968 ident: e_1_2_8_17_1 article-title: Solutions to some functional equations and their applications to characterization of probability distributions publication-title: Sankhyā Ind J Stat Ser A contributor: fullname: Khatri C – ident: e_1_2_8_34_1 doi: 10.1109/TSP.2018.2887192 – ident: e_1_2_8_3_1 – ident: e_1_2_8_8_1 doi: 10.1109/IPDPS.2015.27 – ident: e_1_2_8_18_1 doi: 10.1137/20M1344561 – ident: e_1_2_8_10_1 doi: 10.1109/IPDPS.2017.86 – ident: e_1_2_8_14_1 doi: 10.56021/9781421407944 – ident: e_1_2_8_6_1 doi: 10.1109/IPDPS.2019.00023 – ident: e_1_2_8_30_1 doi: 10.1134/S0965542513120129 – ident: e_1_2_8_2_1 – volume: 20 start-page: 1 issue: 26 year: 2019 ident: e_1_2_8_4_1 article-title: TensorLy: tensor learning in python publication-title: J Mach Learn Res contributor: fullname: Kossaifi J – ident: e_1_2_8_32_1 doi: 10.1038/s41586-022-05172-4 – ident: e_1_2_8_21_1 doi: 10.1145/3432185 – ident: e_1_2_8_25_1 doi: 10.1137/060676489 – ident: e_1_2_8_19_1 doi: 10.1137/120868323 – start-page: 81 volume-title: Data handling in science and technology year: 2019 ident: e_1_2_8_20_1 contributor: fullname: Vervliet N – ident: e_1_2_8_12_1 doi: 10.1007/978-3-319-64203-1_47 – ident: e_1_2_8_31_1 doi: 10.1145/2688500.2688513 – ident: e_1_2_8_9_1 doi: 10.1145/3378445 – ident: e_1_2_8_26_1 doi: 10.1109/HPEC.2012.6408676 – ident: e_1_2_8_7_1 doi: 10.1109/TSP.2013.2269903 – ident: e_1_2_8_11_1 doi: 10.1109/ICPP.2016.19 – ident: e_1_2_8_16_1 doi: 10.1137/07070111X – ident: e_1_2_8_23_1 – ident: e_1_2_8_24_1 doi: 10.1109/IPDPS.2016.113 – ident: e_1_2_8_28_1 doi: 10.1007/BF02165411 – ident: e_1_2_8_15_1 doi: 10.1137/1.9780898719574 – ident: e_1_2_8_27_1 doi: 10.1137/040604959 – ident: e_1_2_8_33_1 doi: 10.1016/0020-0190(79)90113-3 – ident: e_1_2_8_22_1 doi: 10.1109/TSP.2017.2777399 – ident: e_1_2_8_5_1 doi: 10.1137/1.9781611976137.1 |
SSID | ssj0006593 |
Score | 2.3795543 |
Snippet | Abstract
The CP tensor decomposition is used in applications such as machine learning and signal processing to discover latent low‐rank structure in... The CP tensor decomposition is used in applications such as machine learning and signal processing to discover latent low‐rank structure in multidimensional... The CP tensor decomposition is used in applications such as machine learning and signal processing to discover latent low-rank structure in multidimensional... |
SourceID | osti proquest crossref |
SourceType | Open Access Repository Aggregation Database |
SubjectTerms | Algorithms CANDECOMP/PARAFAC canonical polyadic tensor decomposition Decomposition Least squares Machine learning Mathematical analysis MATHEMATICS AND COMPUTING Multidimensional data multilinear algebra numerical stability Singular value decomposition Tensors |
Title | CP decomposition for tensors via alternating least squares with QR decomposition |
URI | https://www.proquest.com/docview/2885374796/abstract/ https://www.osti.gov/servlets/purl/1987855 |
Volume | 30 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3db9MwELegvMADGl-ibCAj8VZlpInjxI9b2ZhQV7qqRX2LnNiZIlUptOke9tfvLnY-yhBfL1HlKFHl3-X8O_vud4R8GHIFLDwQjgeri8NYJh2ZDVMnTLIkxE5JvsZq5MsJv1iwL8tg2SZkVtUlZXKc3v6yruR_UIUxwBWrZP8B2ealMAC_AV-4AsJw_SuMR9OB0pgUbjOvTM4gBKbYQecGy61Wdr-vuMb-ENtysP2xw4ojs_96Ndt_vstUJztzlrMaIBGVmwE2BIHQ2lbDdc69G8zyom7zMUOZ6CZ341sO_6Vq8Y0N7YtmIRjnOxxb5nJddqz0FPf2Tcb95409drb7Ep7_U47Hn72fWYGM20WdUNQ67Pple16Tt072nru38rEreYyRUruk1cf4k6_x-WI8judny_lD8sgLRYAB-qdZKzEGdurXosSu97F-1x5N6a3B3d5brCsGMj8gT23oQE-MHTwjD3TxnDy5bHR3ty_IdDSle4hSsAhqLYICCrRjEbSyCGotgiKu9Gq2__xLsjg_m48uHNszw0nhwyqdBJuJ-yxMmRhmIZdKZ5x7wk0FB287jKRKXK1cFQJ1i3TKlOJachlGiQbunSr_FekV60K_JjQBNudzVynXU0xFMhGCZSJjrBI0clWfvK-nKP5upFFiI4LtVVLlOI19cohzFwOdQ03iFJO30jLGna4oCPrkqJ7S2H5W29iLgEFCkCv4m9_fPiSPW7s7Ir1ys9NvgSGWybsK4jvZ_XA0 |
link.rule.ids | 230,315,786,790,891,27957,27958 |
linkProvider | Wiley-Blackwell |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=CP+decomposition+for+tensors+via+alternating+least+squares+with+QR+decomposition&rft.jtitle=Numerical+linear+algebra+with+applications&rft.au=Minster%2C+Rachel&rft.au=Viviano%2C+Irina&rft.au=Liu%2C+Xiaotian&rft.au=Ballard%2C+Grey&rft.date=2023-12-01&rft.pub=Wiley+Subscription+Services%2C+Inc&rft.eissn=1099-1506&rft.volume=30&rft.issue=6&rft_id=info:doi/10.1002%2Fnla.2511&rft.externalDBID=NO_FULL_TEXT |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1070-5325&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1070-5325&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1070-5325&client=summon |