Data Recovery of Distributed Hash Table with Distributed-to-Distributed Data Copy

• Published in: IEICE Transactions on Information and Systems Vol. E92.D; no. 10; pp. 1999 - 2006
• Main Authors: DOI, Yusuke, WAKAYAMA, Shirou, OZAKI, Satoshi
• Format: Journal Article
• Language: English
• Published: The Institute of Electronics, Information and Communication Engineers 2009
• Subjects: Availability; data durability; Data loss; Data recovery; Distributed Hash Table; Hash based algorithms; Information services; Proposals; Reproduction; scalability; Tables (data); traceability system
• ISSN: 0916-8532; 1745-1361; 1745-1361
• DOI: 10.1587/transinf.E92.D.1999

To realize huge-scale information services, many Distributed Hash Table (DHT) based systems have been proposed. For example, there are some proposals to manage item-level product traceability information with DHTs. In such an application, each entry of a huge number of item-level IDs need to be available on a DHT. To ensure data availability, the soft-state approach has been employed in previous works. However, this does not scale well against the number of entries on a DHT. As we expect 1010 products in the traceability case, the soft-state approach is unacceptable. In this paper, we propose Distributed-to-Distributed Data Copy (D3C). With D3C, users can reconstruct the data as they detect data loss, or even migrate to another DHT system. We show why it scales well against the number of entries on a DHT. We have confirmed our approach with a prototype. Evaluation shows our approach fits well on a DHT with a low rate of failure and a huge number of data entries.