EARLY FUSION OF CAMERA AND RADAR FRAMES

• Main Authors: SUKHAVASI, Ravi Teja, DIJKMAN, Daniel Hendricus Franciscus, MAJOR, Bence, SUBRAMANIAN, Sundar, ANSARI, Amin, GOWAIKAR, Radhika Dilip, LIM, Teck Yian, WU, Xinzhou
• Format: Patent
• Language: English
• Published: 13.06.2024
• Subjects: ANALOGOUS ARRANGEMENTS USING OTHER WAVES; CALCULATING; COMPUTING; CONTROLLING; COUNTING; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; ELECTRIC DIGITAL DATA PROCESSING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION ORRERADIATION OF RADIO WAVES; MEASURING; PHYSICS; RADIO DIRECTION-FINDING; RADIO NAVIGATION; REGULATING; SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES; TESTING

Disclosed are techniques for fusing camera and radar frames to perform object detection in one or more spatial domains. In an aspect, an on-board computer of a host vehicle receives, from a camera sensor of the host vehicle, a plurality of camera frames, receives, from a radar sensor of the host vehicle, a plurality of radar frames, performs a camera feature extraction process on a first camera frame of the plurality of camera frames to generate a first camera feature map, performs a radar feature extraction process on a first radar frame of the plurality of radar frames to generate a first radar feature map, converts the first camera feature map and/or the first radar feature map to a common spatial domain, and concatenates the first radar feature map and the first camera feature map to generate a first concatenated feature map in the common spatial domain.