LARGE MODEL EMULATION BY KNOWLEDGE DISTILLATION BASED NAS
Described herein is a machine learning mechanism implemented by one or more computers, the mechanism having access to a base neural network and being configured to determine a simplified neural network by iteratively performing the following set of steps: forming sample data by sampling the architec...
Saved in:
Main Authors | , , , , |
---|---|
Format | Patent |
Language | English French German |
Published |
12.07.2023
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Described herein is a machine learning mechanism implemented by one or more computers, the mechanism having access to a base neural network and being configured to determine a simplified neural network by iteratively performing the following set of steps: forming sample data by sampling the architecture of a current candidate neural network; selecting, in dependence on the sample data, an architecture for a second candidate neural network; forming a trained candidate neural network by training the second candidate neural network, wherein the training of the second candidate neural network comprises applying feedback to the second candidate neural network in dependence on a comparison of the behaviours of the second candidate neural network and the base neural network; and adopting the trained candidate neural network as the current candidate neural network for a subsequent iteration of the set of steps. |
---|---|
Bibliography: | Application Number: EP20200785967 |