Invariant-Based Automatic Testing of Modern Web Applications

• Published in: IEEE transactions on software engineering Vol. 38; no. 1; pp. 35 - 53
• Main Authors: Mesbah, A., van Deursen, A., Roest, D.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.01.2012; IEEE Computer Society
• Subjects: Ajax; Algorithms; Analysis; Automated testing; Automation; Browsers; Case studies; Computer engineering; Digital Object Identifier; Error detection; Faults; Graphs; Internet; Invariants; Messages; Robots; Servers; Software engineering; Software packages; Studies; Testing; User interface; User interfaces; Web 2.0; Web and internet services; web applications
• ISSN: 0098-5589; 1939-3520
• DOI: 10.1109/TSE.2011.28

Ajax-based Web 2.0 applications rely on stateful asynchronous client/server communication, and client-side runtime manipulation of the DOM tree. This not only makes them fundamentally different from traditional web applications, but also more error-prone and harder to test. We propose a method for testing Ajax applications automatically, based on a crawler to infer a state-flow graph for all (client-side) user interface states. We identify Ajax-specific faults that can occur in such states (related to, e.g., DOM validity, error messages, discoverability, back-button compatibility) as well as DOM-tree invariants that can serve as oracles to detect such faults. Our approach, called Atusa, is implemented in a tool offering generic invariant checking components, a plugin-mechanism to add application-specific state validators, and generation of a test suite covering the paths obtained during crawling. We describe three case studies, consisting of six subjects, evaluating the type of invariants that can be obtained for Ajax applications as well as the fault revealing capabilities, scalability, required manual effort, and level of automation of our testing approach.