Large-scale 3D EM modeling with a Block Low-Rank multifrontal direct solver

• Published in: Geophysical journal international Vol. 209; no. 3; pp. 1558 - 1571
• Main Authors: Shantsev, Daniil, Jaysaval, Piyoosh, de La Kethulle de Ryhove, Sébastien, Amestoy, Patrick, Buttari, Alfredo, L'Excellent, Jean-Yves, Mary, Théo
• Format: Journal Article
• Language: English
• Published: Oxford University Press (OUP) 01.06.2017
• Subjects: Computer Science; Distributed, Parallel, and Cluster Computing; Numerical approximations and analysis; Numerical solutions; Controlled source electromagnetics (CSEM); Numerical modelling; Electromagnetic theory; Marine electromagnetics
• ISSN: 0956-540X; 1365-246X
• DOI: 10.1093/gji/ggx106

We put forward the idea of using a Block Low-Rank (BLR) multifrontal direct solver to efficiently solve the linear systems of equations arising from a finite-difference discretization of the frequency-domain Maxwell equations for 3-D electromagnetic (EM) problems. The solver uses a low-rank representation for the off-diagonal blocks of the intermediate dense matrices arising in the multifrontal method to reduce the computational load. A numerical threshold, the so-called BLR threshold, controlling the accuracy of low-rank representations was optimized by balancing errors in the computed EM fields against savings in floating point operations (flops). Simulations were carried out over large-scale 3-D resistivity models representing typical scenarios for marine controlled-source EM surveys, and in particular the SEG SEAM model which contains an irregular salt body. The flop count, size of factor matrices and elapsed run time for matrix factorization are reduced dramatically by using BLR representations and can go down to, respectively, 10, 30 and 40 per cent of their full-rank values for our largest system with N = 20.6 million unknowns. The reductions are almost independent of the number of MPI tasks and threads at least up to 90 × 10 = 900 cores. The BLR savings increase for larger systems, which reduces the factorization flop complexity from O(N2) for the full-rank solver to O(Nm) with m = 1.4–1.6. The BLR savings are significantly larger for deep-water environments that exclude the highly resistive air layer from the computational domain. A study in a scenario where simulations are required at multiple source locations shows that the BLR solver can become competitive in comparison to iterative solvers as an engine for 3-D controlled-source electromagnetic Gauss–Newton inversion that requires forward modelling for a few thousand right-hand sides.