Deep reinforcement learning for turbulent drag reduction in channel flows

• Published in: The European physical journal. E, Soft matter and biological physics Vol. 46; no. 4; p. 27
• Main Authors: Guastoni, Luca, Rabault, Jean, Schlatter, Philipp, Azizpour, Hossein, Vinuesa, Ricardo
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.04.2023; Springer Nature B.V
• Subjects: Algorithms; Benchmarks; Biological and Medical Physics; Biophysics; Channel flow; Complex Fluids and Microfluidics; Complex Systems; Condensed matter physics; Deep learning; Drag reduction; Flow simulation; Fluid dynamics; Fluid flow; Machine learning; Nanotechnology; Parallel processing; Physics; Physics and Astronomy; Polymer Sciences; Quantitative AI in Complex Fluids and Complex Flows: Challenges and Benchmarks; Regular - Flowing Matter; Regular Article - Flowing Matter; Reynolds number; Soft and Granular Matter; Suction; Surfaces and Interfaces; Thin Films; Turbulence; Turbulent flow
• ISSN: 1292-8941; 1292-895X; 1292-895X
• DOI: 10.1140/epje/s10189-023-00285-8

We introduce a reinforcement learning (RL) environment to design and benchmark control strategies aimed at reducing drag in turbulent fluid flows enclosed in a channel. The environment provides a framework for computationally efficient, parallelized, high-fidelity fluid simulations, ready to interface with established RL agent programming interfaces. This allows for both testing existing deep reinforcement learning (DRL) algorithms against a challenging task, and advancing our knowledge of a complex, turbulent physical system that has been a major topic of research for over two centuries, and remains, even today, the subject of many unanswered questions. The control is applied in the form of blowing and suction at the wall, while the observable state is configurable, allowing to choose different variables such as velocity and pressure, in different locations of the domain. Given the complex nonlinear nature of turbulent flows, the control strategies proposed so far in the literature are physically grounded, but too simple. DRL, by contrast, enables leveraging the high-dimensional data that can be sampled from flow simulations to design advanced control strategies. In an effort to establish a benchmark for testing data-driven control strategies, we compare opposition control, a state-of-the-art turbulence-control strategy from the literature, and a commonly used DRL algorithm, deep deterministic policy gradient. Our results show that DRL leads to 43% and 30% drag reduction in a minimal and a larger channel (at a friction Reynolds number of 180), respectively, outperforming the classical opposition control by around 20 and 10 percentage points, respectively.