Higher efficiency storage replication using compression

In one embodiment, there is a multi-cluster synchronization system between two or more clusters. The multi-cluster synchronization system uses variable compression to optimize the transfer of information between the clusters. Compression is used not only to minimize the total number of bytes sent be...

Full description

Saved in:
Bibliographic Details
Main Authors Gerrard Clay, Barton Michael, Goetz David Patrick, Holt Gregory Lee
Format Patent
LanguageEnglish
Published 31.01.2017
Subjects
CALCULATING
COMPUTING
COUNTING
ELECTRIC COMMUNICATION TECHNIQUE
ELECTRIC DIGITAL DATA PROCESSING
ELECTRICITY
PHYSICS
TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHICCOMMUNICATION
Online AccessGet full text

Cover

More Information
Summary:In one embodiment, there is a multi-cluster synchronization system between two or more clusters. The multi-cluster synchronization system uses variable compression to optimize the transfer of information between the clusters. Compression is used not only to minimize the total number of bytes sent between the two clusters, but to dynamically vary the size of the objects sent across the wire to optimize for higher throughput after considering packet loss, TCP windows, and block sizes. This includes both the packaging of multiple small files together into one larger compressed file, saving on TCP and header overhead, but also the chunking of large files into multiple smaller files that are less likely to have difficulties due to intermittent network congestion or errors. A further embodiment uses forward error correction to maximize the chances that the remote end will be able to correctly reconstitute the transmission.
Bibliography:Application Number: US201414323726