Byzantine-resilient Distributed Large-scale Matrix Completion

• Published in: Proceedings of the ... IEEE International Conference on Acoustics, Speech and Signal Processing (1998) pp. 8167 - 8171
• Main Authors: Lin, Feng, Ling, Qing, Xiong, Zhiwei
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.05.2019
• Subjects: Byzantine attacks; Computational modeling; Data models; Distributed databases; Distributed large-scale optimization; matrix completion; Optimization; Stochastic processes; Task analysis; Training data
• ISSN: 2379-190X
• DOI: 10.1109/ICASSP.2019.8683121

In this paper, we aim at completing a large-scale low-rank matrix over a distributed network, which is subject to Byzantine attacks. We consider solving a nonconvex matrix factorization model with the distributed successive over-relaxation (SOR) method, where the distributed workers compute their private matrices using their own training data and the public matrix sent by the master, while the master updates the public matrix through aggregating the private matrices sent by the workers. However, the Byzantine workers could deliberately send faulty messages to the master so as to bias the optimization process. To address this issue, we propose to replace the aggregation step in the distributed SOR method by several state-of-the-art robust ones: geometric median, median, Krum and h-Krum. We conduct numerical experiments on the Netflix dataset and demonstrate the effectiveness of the proposed robust aggregation strategies in handling Byzantine attacks.