AI-Driven Computer Vision for Detection and Pose Estimation of Chile Peppers (Capsicum annuum L.) for Robotic Harvester — 43p — Amee Parmar, Rishik Aggarwal, Swarnabha Roy

Chili pepper (Capsicum annuum L.) is one of the most widely consumed vegetable crops  worldwide including the United States (U.S.). However, there has been decreasing trend in its  production in the U.S. due to reduced acreage, yields and labor shortages. Therefore, to enhance  U.S. production of chili peppers, phenotypic robot (PhenAI-Bot) and robotic harvester (Rosmaster  X3 Plus) equipped with artificial intelligence (AI) can play critical roles. This study was done as  part of AI algorithm development phase for chili pepper robotic harvester. The objective of this  study was to use YOLOv8 AI model for detecting four varieties of chili peppers (Thai Hot,  Pimiento, Poblano and Serrano) with orientations (horizontal, vertical, and diagonal) that can be  used by the Rosmaster X3 Plus for localization, manipulation and harvesting. For this, 1,234 RGB  images of four varieties of pepper were captured using ELP 4K USB camera. The hyperparameters  (initial and final learning rates, momentum, etc.) of YOLOv8 model were optimized using  mutation genetic algorithm. It was found that without hyperparameter optimization, the mAP50 for  detection and pose estimation tasks was found to be 97.4% and 96.7% respectively while the F1- score for the classification task was found to be 97%. Similarly, after hyperparameter optimization,  the mAP50 for the detection task improved by a little over 0.2% to a value of 97.6% while no  improvements were seen for the rest. The findings from this study suggest that the trained YOLOv8 model can be deployed on the Rosmaster X3 Plus for detection, localization, manipulation and  harvesting of the four varieties of chili peppers in greenhouse settings. The future work will involve  measurements of length and width of each pepper on plants as well as total counts of peppers per plant including greenhouse trails of the Rosmaster X3 Plus for its dexterity, detachment and  harvesting efficiencies. 

South Dakota State University
Dr. Pappu Yadav