What is the weed infestation rate?

Weed infestation rate is an effective quantitative index to measure the degree of weed damage, which helps to quickly calculate the dosage of herbicides and to avoid environmental pollution caused by excessive use of chemicals. In field operations, the weed infestation rate can help to quickly calculate the amount of herbicides and avoid the environmental pollution caused by excessive use of chemicals.

Why PWMD?

PWMD (Pepper Weeds Multimodal Database) is a dataset encompassing field weed data collected through unmanned aerial vehicles, including visible and infrared images. Despite precise weed recognition methods on public datasets, the absence of weed density statistics hampers herbicide quantity determination. Few publicly available non-field image datasets offer annotated details on crop and weed types, impeding weed identification tech progress. To overcome these limitations, we used UAVs with visible and infrared cameras, collecting data at various times, resulting in the creation of PWMD.

Figure 1: The example of the devices and dataset. A: The available multimodal devices for agricultural data acquisition and their operating bands and acquisition platforms. B: Some of the existing near-field weed identification datasets with our collected weed dataset in a pepper field.

Why MWPS?

Weeds significantly impact crop yield, and current weed control equipment struggles to differentiate between crops and weeds, resulting in pesticide wastage and environmental contamination. Accurate herbicide dosing is crucial for effective weed management. Calculating weed infestation rates helps determine herbicide dosage, but traditional visible light equipment is sensitive to light and weather interference, limiting accurate predictions. To enhance weed identification, we introduced supplementary information and utilized a multimodal learning approach, improving model performance. Our weakly supervised weed infestation rate prediction model requires minimal annotations, overcoming limitations of existing models. We propose a multi-model fusion module based on unsupervised learning for quick data processing, suitable for edge devices. Using an unmanned aerial vehicle for data collection, we designed a lightweight and efficient regression model for real-time weed infestation rate prediction. Our multimodal system, based on the Pepper Weeds Multimodal Database (PWMD), addresses data consistency, feature alignment, and effective selection. It includes an image transformation module(ITM) for multimodal data consistency, a multimodal fusion module(MFM) for cross-modal alignment and adaptive fusion, and a prediction module(PM) for improved generalization and rapid, precise weed hazard quantification.

Research Team

SAMLab