Joaquin Gajardo: Wheat 3DGS – In field 3D reconstruction, instance segmentation and phenotyping of wheat heads with gaussian splatting

Bionote

Joaquin Gajardo is a PhD student at ETH Zürich Crop Science group since 2023, working at the intersection of agriculture, computer vision and artificial intelligence. His research focuses on 3D and 4D reconstruction of field crops from multi-view images, with applications to phenotyping and agricultural management. Prior to starting his PhD, he completed his MSc in Environmental Sciences and Engineering at EPFL and worked for two years at EMPA as data scientist on machine learning and remote sensing.

High-throughput field phenotyping at affordable costs is an essential step for sustainable crop production. We present Wheat3DGS, a novel approach for automated wheat head phenotyping in field conditions. We leverage 3D Gaussian Splatting and the Segment Anything Model for precise 3D reconstruction, instance segmentation, and morphological trait extraction of hundreds of wheat heads automatically. Our method can accurately extract and measure length, width, and volume of each wheat head with mean absolute percentage errors of 15.1%, 18.3%, and 40.2% compared to laser scan data. Wheat3DGS enables rapid, non-destructive measurement of key yield-related traits at scale, with significant implications for accelerating crop breeding.

Co-Authors: Daiwei Zhang, Tomislav Medic, Isinsu Katircioglu, Mike Boss, Norbert Kirchgessner, Achim Walter, Lukas Roth