System and method for collecting and annotating cooking images for training smart cooking appliances

• Main Authors: Ma, Jianliang, Liu, Qiang, Yang, Taoli, Yu, Xuemei
• Format: Patent
• Language: English
• Published: 01.10.2024
• Subjects: APPARATUS FOR MAKING BEVERAGES; BLASTING; COFFEE MILLS; CONTROL OR REGULATING SYSTEMS IN GENERAL; CONTROLLING; DETAILS OF DOMESTIC STOVES OR RANGES, OF GENERAL APPLICATION; DOMESTIC ARTICLES OR APPLIANCES; ELECTRIC HEATING; ELECTRIC LIGHTING NOT OTHERWISE PROVIDED FOR; ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR; ELECTRICITY; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; FURNITURE; HEATING; HUMAN NECESSITIES; KITCHEN EQUIPMENT; LIGHTING; MECHANICAL ENGINEERING; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS ORELEMENTS; OTHER DOMESTIC STOVES OR RANGES; PHYSICS; RANGES; REGULATING; SPICE MILLS; SUCTION CLEANERS IN GENERAL; VENTILATING; WEAPONS

The method and system disclosed herein presents a food preparation system that provides a user-friendly data collection and data annotation process that leverages crowd-sourced data to reduce the prohibitive costs associated with developing an extensive training set using dedicated cooking experiments and specialized manual annotation efforts. The food preparation system and corresponding method collect one or more data types (e.g., structured data such as temperature and weight data, and unstructured data such as images and thermal maps) and allow the collected data to be annotated by human users in order to provide information regarding the process of cooking, as well as food item identity, location, and/or outline information. A combination of image processing and user input is utilized in order to create models with food recognition and determining cooking progress.