STATE REPLICATION, ALLOCATION AND FAILOVER IN STREAM PROCESSING

• Main Authors: Venkatraman, Jagadish, Venkataraman, Shanthoosh Pazhanjur, Matharu, Ray Manpreet Singh, Kumarasubramanian, Bharath, Pan, Yi, Shetty, Samarth, Maheshwari, Prateek, Paramasivam, Kartik, Maes, Jacob R
• Format: Patent
• Language: English
• Published: 10.12.2020
• Subjects: ELECTRIC COMMUNICATION TECHNIQUE; ELECTRICITY; TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHICCOMMUNICATION

The disclosed embodiments provide a system for managing a stream-processing application. During operation, the system allocates a first host for an active instance of the stream-processing application that maintains a state during processing of one or more input streams. Next, the system allocates a second host for a first backup instance that recreates the state on the active instance by consuming changes to the state replicated from the active instance without consuming the input stream(s). During a failure on the first host, the system moves the active instance to the first host by stopping the first backup instance on the second host. Finally, the system launches the active instance on the second host to resume processing of the input stream(s) by the active instance using the recreated state from the first backup instance.