Improving Inference Performance of Machine Learning with the Divide-and-Conquer Principle

• Published in: arXiv.org
• Main Author: Kogan, Alex
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 01.03.2023
• Subjects: Inference; Machine learning; Natural language processing; Optical character recognition; Principles

Many popular machine learning models scale poorly when deployed on CPUs. In this paper we explore the reasons why and propose a simple, yet effective approach based on the well-known Divide-and-Conquer Principle to tackle this problem of great practical importance. Given an inference job, instead of...