Bionote
Jianchao Ci is a final-year Ph.D. student in Agricultural Biosystems Engineering at Wageningen University, specializing in learning-based active perception for agri-food robotics. His research focuses on developing efficient 3D plant reconstruction methods to automate agricultural tasks like harvesting and phenotyping, with publications in Biosystems Engineering and Computers and Electronics in Agriculture.
He holds an M.Sc. in Biosystems Engineering from Wageningen University (2021) and has professional experience as an algorithm engineer in computer vision at Rokae Robotics in Beijing. His technical expertise includes deep learning, robotic control, view planning, and machine vision.
Contact: jianchao.ci@wur.nl
Presentation Abstract
The 3D reconstruction of plants is challenging due to their complex shape causing many occlusions. Next-Best-View (NBV) methods address this by iteratively selecting new viewpoints to maximize information gain (IG). Deep-learning-based NBV (DL-NBV) methods demonstrate higher computational efficiency over classic voxel-based NBV approaches but current methods require extensive training using ground-truth plant models, making them impractical for real-world plants. These methods, moreover, rely on offline training with pre-collected data, limiting adaptability in changing agricultural environments. This paper proposes a self-supervised learning-based NBV method (SSL-NBV) that uses a deep neural network to predict the IG for candidate viewpoints. The method allows the robot to gather its own training data during task execution by comparing new 3D sensor data to the earlier gathered data and by employing weakly-supervised learning and experience replay for efficient online learning.
Comprehensive evaluations were conducted in simulation and real-world environments using cross-validation. The results showed that SSL-NBV required fewer views for plant reconstruction than non-NBV methods. It achieved IG prediction in 0.0038s, making it over 800 times faster than a voxel-based NBV, and an online learning iteration in 0.099s. SSL-NBV reduced training annotations by over 90% compared to a baseline DL-NBV. Furthermore, SSL-NBV could adapt to novel scenarios through online fine-tuning. Also using real plants, the results showed that the proposed method can learn to effectively plan new viewpoints for 3D plant reconstruction. Most importantly, SSL-NBV automated the entire network training and uses continuous online learning, allowing it to operate in changing agricultural environments.
Co-Authors: Eldert J. van Henten, Xin Wang, Gert Kootstra