Method and device for data deduplication based on K-V storage engine

The invention discloses a method and a device for data deduplication based on a K-V storage engine. The method comprises the following steps of splitting a file into data blocks with the fixed length; carrying out Hash computation on each data block to obtain a data block abstract array; and compari...

Full description

Saved in:
Bibliographic Details
Main Authors ZHANG YAO, NI SHILONG, SU JIANGWEN
Format Patent
LanguageChinese
English
Published 05.04.2017
Subjects
CALCULATING
COMPUTING
COUNTING
ELECTRIC DIGITAL DATA PROCESSING
PHYSICS
Online AccessGet full text

Cover

More Information
Summary:The invention discloses a method and a device for data deduplication based on a K-V storage engine. The method comprises the following steps of splitting a file into data blocks with the fixed length; carrying out Hash computation on each data block to obtain a data block abstract array; and comparing abstracts in the array with a data block abstract stored in the K-V storage engine one by one, if the same abstract exists, not carrying out storage, otherwise storing into the K-V storage engine with the data block abstract as a Key and a to-be-stored data block as a Value. 种基于K-V存储引擎的数据去重方法及装置，其中方法包括如下步骤，将文件分割成定长的数据块，对各数据块进行哈希计算，得到数据块摘要数组，将数组中摘要逐个与已存储在K-V存储引擎中的数据块摘要进行比对，若存在相同的摘要，则不进行存储，否则以数据块摘要为Key，待存储数据块为Value存入K-V存储引擎。
Bibliography:Application Number: CN201611034583