FEDERATED COMPUTATIONAL ANALYSIS OVER DISTRIBUTED DATA

• Main Authors: CHATZOU, Maria, SOSIC, Martin, SILVA, Diogo, Nuno, Proenca, DE JESUS, Tiago, Filipe, Salgueiro, GONCALVES, Bruno, Filipe, Ribeiro, DOBREV, Damyan, SOSIC, Matija, KRUGLOVA, Olga, BARJA, Pablo, Prieto
• Format: Patent
• Language: English; French; German
• Published: 26.10.2022
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

The present disclosure provides for computational data analysis across multiple data sources. A pipeline (or workflow) is imported and a dataset is selected. The dataset resides on a virtual file system and includes data residing on one or more storage locations associated with the virtual file system. One or more compute resources are selected to perform the pipeline analysis based at least on the imported pipeline and the dataset. The one or more compute resources are selected from a plurality of available compute resources associated with the one or more storage locations associated with the virtual file system. The pipeline analysis is performed using the selected compute resources on the dataset in one or more secure clusters. The resulting data generated from the pipeline analysis is submitted to the virtual file system.