Challenges

A recent publication  by Müllerová, et al., (2023), provides a full review of the use of remote sensing of weeds and the many limitations that are still being overcome in this field, entitled:

Müllerová, J., Brundu, G., Große-Stoltenberg, A., Kattenborn, T., & Richardson, D. M. (2023). Pattern to process, research to practice: remote sensing of plant invasions. Biological Invasions, 1-26.

A number of points outlined in this review have been used in the discussion below.

Despite its significant advantages, remote sensing for weed detection has inherent challenges and  limitations that can impact its effectiveness. One key limitation lies in the potential for misclassification and misinterpretation of spectral signatures. The spectral reflectance patterns of weeds may overlap with those of other vegetation or background elements, leading to false positives or negatives. Additionally, variations in environmental conditions, such as lighting and atmospheric interference, can introduce uncertainties in the acquired data, further challenging the accuracy of weed detection algorithms. Plants are also often required to be mapped within a specific period of time associated with plant development (e.g., flowering), or indirectly by identifying structural changes, or in detecting shades and green vegetation. At times, detection may only be achievable to the genus level, since many species appear visually similar.

Another core disadvantage is the limited spatial resolution of some remote sensing systems. While satellite imagery provides extensive coverage, it may lack the level of detail required for precise weed mapping in smaller agricultural plots. This can be particularly problematic when dealing with weed species that exhibit subtle morphological differences. The compromise between spatial and spectral resolution is an ongoing challenge in remote sensing applications, impacting the ability to discriminate between closely spaced or small weed infestations accurately. Overall, these limitations highlight the need for careful consideration and integration of ground-truthing methods and complementary technologies to enhance the reliability of remote sensing for weed detection.

The use of various spatial or temporal scales can be effective in increasing detection success and these can also be used in combination. However, accuracy can decline due to reduced phenological separation of plants and greater ROI complexity when a wide-angle camera view is used  and when comparing different sites, where vegetation can be vastly different.

Advanced sensors such as hyperspectral cameras, or the combination of several different sensors, can also be used to increase detection success between species. Further, photogrammetric digital surface models, created as part of the drone imagery processing pipeline, are also valuable for improving accuracy.

Capturing temporal changes in weed invasions still remains a challenge in weed remote detection. This is due to the variability in spectral reflectance of invasive plants during phenological development, which can hinder the ability of trained algorithms to detect species in different locations and across time. This also makes it difficult to use historical data. Furthermore, it is usually only expedient for highly distinguishable species and where adequate time series data has been generated.  In a weed detection ROI containing large areas of plant species variability, temporal changes also affect the adjacent and surrounding plants within the area. This means that additional temporal data capture is often  needed for the process of elimination,  in addition to the use of detection algorithms. In some cases, the rich abundance of the data can alleviate spatial resolution deficiencies, particularly when the target plant phenology is vastly different from the surrounding species, but this can be costly to attain.

Limitations also exist during the data capture process. Most drone platforms have a short flight time and require extensive battery recharge to be able to fly larger areas. In remote regions this can be a challenge and time and extra batteries will need to be considered during the mission planning period. There are also limitations placed on drone missions as a result of weather, where light and wind conditions must be optimal to ensure high imagery quality whilst at the site. This can be difficult to plan for in some regions. Suboptimal weather tends to increase the cost of imagery capture, since the flight operator may be required to wait, sometimes for a day or more, before more optimal flying conditions become available to complete the mission. This may be particularly problematic in remote areas and for some weed species, since the optimal flying time may misalign with optimal phenological development for detection. This is the case particularly for species like Hawkweed, which flowers across a very short window over summer in Australia.

Likewise, the terrain can present challenges in accessing sites and can present constraints in finding a suitable drone take off/landing  area.  If some species are located on cliff sides or on very steep terrain, it can be difficult to capture clear imagery from above (referred to as “Nadir”). In these cases, a more side-on capture may be required (“Off-Nadir” or oblique), but this will influence the performance of models later down the pipeline and labelling and parameters will need adjustment to account for the different angle of view.

When carrying out weed detection missions, it is important to note that there are usually delays between data acquisition, processing, analysis and delivery of detection maps to weed managers. For successful detection using deep learning algorithms, there must be a significant number of images captured that are labelled for the training dataset. This can present cost challenges to a team capturing the imagery and this can be exacerbated by the unique challenges described above, including poor weather, difficult terrain and access to sites. Further, there are limitations in the use of models when datasets are small, where validations of the models have been minimal and in terms of their transferability across species, time, missions and sites. Models need to be precise and instructional as suitable for the particular management goal of the species. For example, for some species that may be eradication targets, it will be paramount to reduce the number of false positives and false negatives close to zero and accuracy must be higher than 95% since every plant must be identified. This is the case for hawkweeds in Australia, which are eradication targets. For this to be possible, significant amounts of labelled data will be required and the highest resolution possible within budget limitations. However, due to the cost of capturing high resolution imagery over such large areas, lower resolution imagery has become a more desirable option. Using lower resolution imagery for accurate detection will only be possible with vast amounts of training data to correctly train the models, since it is more difficult to distinguish target plants in lower resolution imagery.

Alternatively for other species,  lower accuracy (and hence a degree of false positives and negatives) may be more acceptable since the imagery will be simply used to identify where a species might be located in the landscape, so that the land manager knows where to visit and identify particular plants or stands as part of routine control. This may be the case for weeds like Bitou Bush and African Lovegrass, which often take up significant areas in a site and once found as present, require ground based efforts to identify and control the rest in that area.

In any case, it will always be important to report any discrepancies in data and limitations of the methods. It will also be important to explain the detection potential of the models  so that they can be used correctly and understood by the various stakeholders who may use them. This builds trust in remote sensing approaches and facilitates ease of access and implementation.

One of the other disadvantages of using remote sensing for weed detection is the long term developmental process required to develop models, suitable platforms and interfaces for use by  field users. There has also been a gap in the availability  of national standards and a suitable platform for sharing data of this nature. The Australian Scalable Drone Cloud (Australian National University Plant Phenomics Facility) is an example of a technological advancement that is undergoing significant development to move towards this space, with the aim of generating a national platform for drone data and enabling cloud-based processing and weed detection at the ground level across Australia.

Historically, other challenges faced by those engaging in weed remote detection have included:

• Impractical access to processing and remote sensing platforms by field personnel
• Differences in the mindset of different weed control groups
• Time lags - visually observing the weeds when they are a problem is usually when flight decisions occur, however they often require decisions and control at a much earlier level, before they are able to be detected. The process of developing weed detection pipelines that are accessible are often 10-15 years in development time. However, weed detection processes often require a much more immediate response.
• There is often an inaccurate perception of what UAVs can and cannot do in the current environment.