The Weed Manager’s Guide To Remote Detection
The Weed Manager’s Guide To Remote Detection
DPIRD has carried out an initial proof-of-concept phase to determine that Skeleton Weed infestations can be detected with AI. However, it has determined that a much bigger dataset and involvement of weed identification Subject Matter Experts (SMEs) is crucial for high-quality datasets and robust models.
Partners: DPIRD, IFS (Grain Seeds and Hay), Mortlock Local Action Group Coordinator (Katie Chester), Sensorem
Furthermore, Deep Learning AI models performing the task of Object Detection are to be used instead of tile classification for higher model recall. In-house (local machine) inference has been the preference to avoid high costs of cloud processing with large models on large images.
RGB imagery has been used both at 61 and 45MP with Sony a7R IV and Zenmuse P1 cameras mounted on DJI M300. The Ultralytics python package has been used to perform predictions and custom code for project and data management with Label Studio SDK facilitated the involvement of SME for data validation.
Surveillance for skeleton weed in WA typically takes place at least 10 days post-harvest to allow the plants to present themselves as deep green branches in a amongst dry stubble.
Fire Danger Rating and Vehicle bans typical of summertime are a constant cause of setback of the surveillance program.
Furthermore, due to the risk of starting a fire, at least one of the vehicles must be equipped with fire-fighting equipment and at least 400L of water.
The typical coverage of surveillance for full search is about 200 ha per day with two spotters in a search unit and about 300 ha per day for surveillance search operations.
At this stage, drones can match the efficiency of surveillance search operations at around 300ha per day and around the same cost per hectare also.
There are several advantages of Aerial surveillance in comparison to ground-based operations during summer:
The latest Program Review in 2020 (Bowran et al., 2020) found that “New surveillance technologies to identify new infestations will be critical to finding plants before infestations spread seeds widely. […]” Where “[…] An area of 1.8 million hectares would need to be surveyed to understand the full extent of skeleton weed presence in WA […]” and “[…] While investment has been seen in drone (UAV) technology and neural networks software capable of identifying skeleton weed in WA, a much larger image dataset is required.”
Initial work to be able to capture images of the plants with enough resolution to distinguish them from the background was a first step. However, outsourcing the labelling task proved challenging, as identifying the plants from look-alikes was a task for experts in weed identification.
Ground truthing individual plants with centimetre precision was a game changer to collate a high-quality dataset. However, SME involvement on detections for Human-in-the-loop (HITL) integration only came to be realised at a later stage.
A wider footprint of the surveillance has been achieved and the evolving datasets, models, and HITL system set in place will provide the ability to continue increasing the surveillance footprint to aid ground surveillance and the objectives of the program.
Lessons learned will be presented at the 23rd Australasian Weeds Conference (AWC) August 2024
The considerations to take into account for collecting the right type of data, in the right way, as well as the mechanisms used to allow weed experts as SMEs to be intrinsically involved in the development and curation of high-quality datasets. The schematic approach will be made available along with the AWC presentation.
Fig.1. Schematic view of cm-precision mapping equipment used for ground truth of Skeleton Weed surveys in WA, and the resulting initial georeferenced image-tile index with plant annotations.
Fig.2. Screenshot - side-by-side imagery footprint overlapping ground truth areas of interest and Label Studio interface where HITL takes place: SME validates detections, discards false positives, and labels false negatives to increase validated dataset at a large scale.