NTrack: A Multiple-Object Tracker and Dataset for Infield Cotton Boll Counting

• Published in: IEEE transactions on automation science and engineering Vol. 21; no. 4; pp. 1 - 13
• Main Authors: Muzaddid, Md Ahmed Al, Beksi, William J.
• Format: Journal Article
• Language: English
• Published: IEEE 20.12.2023
• Subjects: Agricultural automation; computer vision for automation; visual tracking
• ISSN: 1545-5955; 1558-3783
• DOI: 10.1109/TASE.2023.3342791

In agriculture, automating the accurate tracking of fruits, vegetables, and fiber is a very tough problem. The issue becomes extremely challenging in dynamic field environments. Yet, this information is critical for making day-to-day agricultural decisions, assisting breeding programs, and much more. To tackle this dilemma, we introduce NTrack , a novel multiple object tracking framework based on the linear relationship between the locations of neighboring tracks. NTrack computes dense optical flow and utilizes particle filtering to guide each tracker. Correspondences between detections and tracks are found through data association via direct observations and indirect cues, which are then combined to obtain an updated observation. Our modular multiple object tracking system is independent of the underlying detection method, thus allowing for the interchangeable use of any off-the-shelf object detector. We show the efficacy of our approach on the task of tracking and counting infield cotton bolls. Experimental results show that our system exceeds contemporary tracking and cotton boll-based counting methods by a large margin. Furthermore, we publicly release the first annotated cotton boll video dataset to the research community. Note to Practitioners -This work is motivated by the need to provide highly-accurate estimates of the total number of cotton bolls across an entire farm. We provide a multiple object tracking framework for automating the counting process using a dynamic motion model that can reidentify severely occluded objects. This information is immensely beneficial to agronomists and breeders. For example, it can allow them to accelerate the selection of genotypes and identify cotton cultivars that exhibit tolerance to adverse environmental conditions (e.g., drought, poor soil quality, etc.) via yield prediction. Since the performance of our tracker is tied to the accuracy of the object detector, the ability to swap detectors is important for enhancing the usability of the system. Our dataset structure was modeled after similar multiple object tracking datasets for the purpose of increasing adoption among practitioners. To implement our system, it is assumed that the image/video capture device is of high resolution and data is acquired under ideal lighting conditions. The framework can be run on any commercial-off-the-shelf platform (e.g., ground-based robots, unmanned aerial vehicles, etc.), with sufficient computing and memory resources, using a vision-based sensor.