Private Gradient Descent for Linear Regression: Tighter Error Bounds and Instance-Specific Uncertainty Estimation
We provide an improved analysis of standard differentially private gradient descent for linear regression under the squared error loss. Under modest assumptions on the input, we characterize the distribution of the iterate at each time step. Our analysis leads to new results on the algorithm's...
Saved in:
Published in | arXiv.org |
---|---|
Main Authors | , , , , , |
Format | Paper |
Language | English |
Published |
Ithaca
Cornell University Library, arXiv.org
21.02.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | We provide an improved analysis of standard differentially private gradient descent for linear regression under the squared error loss. Under modest assumptions on the input, we characterize the distribution of the iterate at each time step. Our analysis leads to new results on the algorithm's accuracy: for a proper fixed choice of hyperparameters, the sample complexity depends only linearly on the dimension of the data. This matches the dimension-dependence of the (non-private) ordinary least squares estimator as well as that of recent private algorithms that rely on sophisticated adaptive gradient-clipping schemes (Varshney et al., 2022; Liu et al., 2023). Our analysis of the iterates' distribution also allows us to construct confidence intervals for the empirical optimizer which adapt automatically to the variance of the algorithm on a particular data set. We validate our theorems through experiments on synthetic data. |
---|---|
AbstractList | We provide an improved analysis of standard differentially private gradient descent for linear regression under the squared error loss. Under modest assumptions on the input, we characterize the distribution of the iterate at each time step. Our analysis leads to new results on the algorithm's accuracy: for a proper fixed choice of hyperparameters, the sample complexity depends only linearly on the dimension of the data. This matches the dimension-dependence of the (non-private) ordinary least squares estimator as well as that of recent private algorithms that rely on sophisticated adaptive gradient-clipping schemes (Varshney et al., 2022; Liu et al., 2023). Our analysis of the iterates' distribution also allows us to construct confidence intervals for the empirical optimizer which adapt automatically to the variance of the algorithm on a particular data set. We validate our theorems through experiments on synthetic data. |
Author | Smith, Adam Evans, Georgina Liu, Daogao Brown, Gavin Thakurta, Abhradeep Krishnamurthy Dvijotham |
Author_xml | – sequence: 1 givenname: Gavin surname: Brown fullname: Brown, Gavin – sequence: 2 fullname: Krishnamurthy Dvijotham – sequence: 3 givenname: Georgina surname: Evans fullname: Evans, Georgina – sequence: 4 givenname: Daogao surname: Liu fullname: Liu, Daogao – sequence: 5 givenname: Adam surname: Smith fullname: Smith, Adam – sequence: 6 givenname: Abhradeep surname: Thakurta fullname: Thakurta, Abhradeep |
BookMark | eNqNjNFKw0AQRRexYLX9h4E-B9Zd20Qfq6kKPojW57Ikk3aKzrYzk4J_bwQ_wKfD4R7upTvnzHjmxiHG66K6CeHCTVX33vuwKMN8Hsfu-Cp0SobwKKklZIMH1OaXXRZ4IcYk8IZbQVXKfAdr2u4MBWqRIVjmnluFxC08s1riBov3AzbUUQMfg4klYvuGWo2-kg0XEzfq0qfi9I9Xbraq1_dPxUHysUe1zT73wsO0CbfR-8qXVRn_V_0Aw1VNEg |
ContentType | Paper |
Copyright | 2024. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
Copyright_xml | – notice: 2024. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central (Alumni) ProQuest Central ProQuest Central Essentials ProQuest Central Technology Collection ProQuest One Community College ProQuest Central Korea SciTech Premium Collection ProQuest Engineering Collection Engineering Database Publicly Available Content (ProQuest) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection |
DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest One Academic Engineering Collection |
DatabaseTitleList | Publicly Available Content Database |
Database_xml | – sequence: 1 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Physics |
EISSN | 2331-8422 |
Genre | Working Paper/Pre-Print |
GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
ID | FETCH-proquest_journals_29300807873 |
IEDL.DBID | 8FG |
IngestDate | Thu Oct 10 15:54:21 EDT 2024 |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-proquest_journals_29300807873 |
OpenAccessLink | https://www.proquest.com/docview/2930080787?pq-origsite=%requestingapplication% |
PQID | 2930080787 |
PQPubID | 2050157 |
ParticipantIDs | proquest_journals_2930080787 |
PublicationCentury | 2000 |
PublicationDate | 20240221 |
PublicationDateYYYYMMDD | 2024-02-21 |
PublicationDate_xml | – month: 02 year: 2024 text: 20240221 day: 21 |
PublicationDecade | 2020 |
PublicationPlace | Ithaca |
PublicationPlace_xml | – name: Ithaca |
PublicationTitle | arXiv.org |
PublicationYear | 2024 |
Publisher | Cornell University Library, arXiv.org |
Publisher_xml | – name: Cornell University Library, arXiv.org |
SSID | ssj0002672553 |
Score | 3.5228453 |
SecondaryResourceType | preprint |
Snippet | We provide an improved analysis of standard differentially private gradient descent for linear regression under the squared error loss. Under modest... |
SourceID | proquest |
SourceType | Aggregation Database |
SubjectTerms | Algorithms Confidence intervals Empirical analysis Statistical analysis Synthetic data |
Title | Private Gradient Descent for Linear Regression: Tighter Error Bounds and Instance-Specific Uncertainty Estimation |
URI | https://www.proquest.com/docview/2930080787 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1LS8NAEB60QfDmEx-1DOh10WyalxehmrQKLaW00FvJZjfiJW038eDF3-7sNtWD0OOyIWTD7HzfzH6zA3BHkb_iXizZg_R9Roy4YDGhBBPdIOM8FDLKTIHzcBQMZt23uT9vEm5VI6vc-kTrqOUyNznye4Ilw27Ivp5Wa2a6RpnT1aaFxj44Lg8DI-mL0v5vjoUHITFm75-btdiRHoEzzlZKH8OeKk_gwEou8-oU1mNtOosp7Gsru6rxZXO1EhKPRIoRyQZxot43QtXyEacmjlYaE63pgZ7ph1RhVkp8tRQvV8w2ky8-cpzRyB7111-Y0Cbe1CeewW2aTJ8HbPudi8aSqsXfur1zaJXLUl0AKhnyws3dQqqIQhYuBG1Lzy84F1zErncJ7V1vuto9fQ2HnKDbFm67bWjV-lPdEPTWomP_bwecXjIaT2g0_E5-AM4ukBo |
link.rule.ids | 780,784,12765,21388,33373,33744,43600,43805 |
linkProvider | ProQuest |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV07T8MwED5BKwQbT_EoYAlWC-K8GhYkIGkKbVWhVOoWxbGDWNLWCQP_nrOTwoDU0YplJdH5vu_Od_4AbjHyl8wOBL0XrkuRERc0QJSg3PEyxnwu-plucB5PvHjmvM7deZtwq9qyyrVPNI5aLHKdI79DWNLsBu3rcbmiWjVKn662Ehrb0HVshG7dKR4NfnMszPORMdv_3KzBjmgfutNsKdUBbMnyEHZMyWVeHcFqqrSymCQDZcquavLSXK1EkEcSjBHRBsm7_GgKVcsHkug4WioSKoUTnrQeUkWyUpChoXi5pEZMvvjMyQxH5qi__iYhbuKmP_EYbqIweY7p-j3T1pKq9O-77RPolItSngKRwmeFlVuFkH0MWRjnuC1tt2CMMx5Y9hn0Nq10vvnxNezGyXiUjoaTtwvYYwjjponb6kGnVl_yEmG45lfmX_8AuC6QMQ |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Private+Gradient+Descent+for+Linear+Regression%3A+Tighter+Error+Bounds+and+Instance-Specific+Uncertainty+Estimation&rft.jtitle=arXiv.org&rft.au=Brown%2C+Gavin&rft.au=Krishnamurthy+Dvijotham&rft.au=Evans%2C+Georgina&rft.au=Liu%2C+Daogao&rft.date=2024-02-21&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422 |