Complexity-based deep learning model generalization estimation method

• Main Authors: SHENG XIAOJUAN, LIU JIAJUN, WANG PENG, LIU SHAOTONG, KE WENJUN, LEE SUNUL, WEN WANTAO
• Format: Patent
• Language: Chinese; English
• Published: 02.08.2024
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; PHYSICS

A deep learning model generalization estimation method based on complexity comprises the steps that firstly, a model structure complexity index and a model parameter norm complexity index are considered, the expectation sharpness of a model local minimum solution is measured in combination with a PAC-Bayes theory, a complexity index based on the sharpness of a model solution is proposed, and the measurement object is a convolutional neural network of an image classification task; then, based on the complexity index set constructed by the patent, several common machine learning models SVR, RF, XGBoost and FCNN are adopted to establish regression models for model complexity and generalization differences; in order to solve the problem that hyper-parameters of a machine learning model are difficult to design manually, the hyper-parameters are optimized by adopting Bayesian optimization. And finally, regression effects of several machine learning models after hyper-parameter tuning are analyzed, and the model wit