Scaling Web API Integrations

• Published in: 2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) pp. 13 - 23
• Main Authors: Chari, Guido, Sheffer, Brandon, Branavan, S. R. K, D'ippolito, Nicolas
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.05.2023
• Subjects: Codes; Companies; Computer architecture; n/a; Natural language processing; Ontologies; Planning; Software
• ISSN: 2832-7659
• DOI: 10.1109/ICSE-SEIP58684.2023.00007

In ASAPP, a company that offers AI solutions to enterprise customers, internal services consume data from our customers' web APIs. Implementing and maintaining integrations between our customers' APIs and internal services is a major effort for the company. In this paper, we present a scalable approach for integrating web APIs in enterprise software that is lightweight and semi-automatic. It leverages a combination of Ontology-Based Data Access architectures (OBDA), a Domain Specific Language (DSL) called IBL, Natural Language Processing (NLP) models, and Automated Planning techniques. The OBDA architecture decouples our platform from our customers' APIs via an ontology that acts as a single internal data access point. IBL is a functional and graphical DSL that enables domain experts to implement integrations, even if they don't have software development expertise. To reduce the effort of manually writing the IBL code, an NLP model suggests correspondences from each web API to the ontology. Given the API, ontology, and selected mappings for a set of desired fields from the ontology, we define an Automated Planning problem. The resulting policy is finally fed to a code synthesizer that generates the appropriate IBL method implementing the desired integration.This approach has been in production in ASAPP for 2 years with more than 300 integrations already implemented. Results indicate a ≈ 50% reduction in effort due to implementing integrations with IBL. Preliminary results on the IBL automatic code generation show an encouraging further ≈ 25% reduction so far.