DATA INGESTION USING FILE QUEUES

The subject technology at a data system, an ingest request to ingest one or more files into a table. The subject technology, after obtaining the ingest request and prior to the ingesting of the one or more files, persists the one or more files in a first file queue that corresponds to the table, the...

Full description

Saved in:
Bibliographic Details
Main Authors Dageville, Benoit, Ganesh, Varun, Huang, Jiansheng, Yu, Haowei, Ziegler, Scott, Liang, Jiaxing
Format Patent
LanguageEnglish
Published 12.11.2020
Subjects
Online AccessGet full text

Cover

More Information
Summary:The subject technology at a data system, an ingest request to ingest one or more files into a table. The subject technology, after obtaining the ingest request and prior to the ingesting of the one or more files, persists the one or more files in a first file queue that corresponds to the table, the first file queue further corresponding to a client account, and the data system further comprising a second file queue that corresponds to both a second client account and a second table. The subject technology ingests, by one or more execution nodes, the one or more files into one or more micro-partitions of the table, each of the one or more micro-partitions comprising contiguous units of storage of a storage device.
Bibliography:Application Number: US202016943251