Transformer Soil Mask

Converts an RGB image into a soil mask in which the soil is represented as black.

The core idea for this Transformer is a plant-soil segmentation that was described by Li et al 2019.

Algorithm Description

The core idea for this Transformer is a plant-soil segmentation. We apply a threshold to differentiate plant and soil, and do a smoothing after binary processing. Saturated portions of the image are removed. At the end, it returns the plant area ratio (canopy cover) within a bounding box.

Steps:

1. Split image data into R,G,B channel, and make a tmp image.
2. For each pixel, if G value is T(threshold) higher than R value, make this pixel as foreground, and set the tmp pixel value to 255, so all tmp pixels are 0 or 255.
3. Use a filter to blur this tmp image
4. Remove anomalies (small areas incorrectly classified as plant of interest)
5. Threshold the blurred tmp image with a threshold of 128 to get a new mask image that represents our plant (foreground) detections.
6. Remove saturated pixels
7. Output ratio = foreground pixel count / total pixel count

Parameters

• G - R Threshold is set to 2 for normal situation.
• Blur: image to new mask threshold is set to 128; passed to the OpenCV blur function.
• Saturation threshold: threshold for classifying a pixel as saturated. Default is 245 in a greyscale imagess
• Small Area Threshold: Used to remove anomalies from the image - this parameter is the size of a mask fragment in pixels that is removed.

Quality Statement

Currently, this algorithm has been used on wheat and sorghum; it has been tested on lettuce but only works when the leaves are green (fails if they are red or purple).

We believe the tested threshold works well in a normal illumination. Below are three examples of successful segmentation:

At the same time, there are some limitations with the current threshold. Here are some examples:

1. Image captured in a low illumination.

2. Image captured in a very high illumination.

3. In late season, panicle is covering a lot in the image, and leaves is getting yellow.

4. Sometimes an unidentified sensor problem results in a blank image.

For more details, see related discussions, including: terraref/reference-data#186 (comment)

Li, Zongyang, Abby Stylianou, and Robert Pless. "Learning to Correct for Bad Camera Settings in Large Scale Plant Monitoring." CVPPP 2019 Computer Vision Problems in Plant Phenotyping.

Use

Sample Docker Command line

First build the Docker image, using the Dockerfile, and tag it agdrone/transformer-soilmask:2.4. Read about the docker build command if needed.

docker build -t agdrone/transformer-soilmask:2.4 ./

There are two files needed for running the Docker image. In the example below the experiment.yaml file contains information on the experiment. The orthomosiac.tif file is an Orthomosaic image that is to have the soil removed. These two files can be retrieved using the following commands:

mkdir test_data
curl -X GET https://de.cyverse.org/dl/d/3C8A23C0-F77A-4598-ADC4-874EB265F9B0/scif_test_data.tar.gz -o test_data/scif_test_data.tar.gz
tar -xzvf test_data/scif_test_data.tar.gz -C test_data/

Below is a sample command line that shows how the soil mask Docker image could be run. An explanation of the command line options used follows. Be sure to read up on the docker run command line for more information.

docker run --rm --mount "src=${PWD}/test_data,target=/mnt,type=bind" agdrone/transformer-soilmask:2.4 --working_space "/mnt" --metadata "/mnt/experiment.yaml" "/mnt/orthomosaic.tif"

This example command line assumes the source files are located in the test_data folder off the current folder. The name of the image to run is agdrone/transformer-soilmask:2.4.

We are using the same folder for the source files and the output files. By using multiple --mount options, the source and output files can be separated.

Docker commands
Everything between 'docker' and the name of the image are docker commands.

• run indicates we want to run an image
• --rm automatically delete the image instance after it's run
• --mount "src=${PWD}/test_data,target=/mnt,type=bind" mounts the ${PWD}/test_data folder to the /mnt folder of the running image

We mount the ${PWD}/test_data folder to the running image to make files available to the software in the image.

Image's commands
The command line parameters after the image name are passed to the software inside the image. Note that the paths provided are relative to the running image (see the --mount option specified above).

• --working_space "/mnt" specifies the folder to use as a workspace
• --metadata "/mnt/experiment.yaml" is the name of the source metadata
• "/mnt/orthomosaic.tif" is the name of the image to mask

Acceptance Testing

There are automated test suites that are run via GitHub Actions. In this section we provide details on these tests so that they can be run locally as well.

These tests are run when a Pull Request or push occurs on the develop or main branches. There may be other instances when these tests are automatically run, but these are considered the mandatory events and branches.

PyLint and PyTest

These tests are run against any Python scripts that are in the repository.

PyLint is used to both check that Python code conforms to the recommended coding style, and checks for syntax errors. The default behavior of PyLint is modified by the pylint.rc file in the Organization-info repository. Please also refer to our Coding Standards for information on how we use pylint.

The following command can be used to fetch the pylint.rc file:

wget https://raw.githubusercontent.com/AgPipeline/Organization-info/main/pylint.rc

Assuming the pylint.rc file is in the current folder, the following command can be used against the soilmask.py file:

# Assumes Python3.7+ is default Python version
python -m pylint --rcfile ./pylint.rc soilmask.py

In the tests folder there are testing scripts; their supporting files are in the test_data folder. The tests are designed to be run with Pytest. When running the tests, the root of the repository is expected to be the starting directory.

These tests use some of the files downloaded from CyVerse. The following commands download and extracts the files in this archive:

curl -X GET https://de.cyverse.org/dl/d/3C8A23C0-F77A-4598-ADC4-874EB265F9B0/scif_test_data.tar.gz -o test_data/scif_test_data.tar.gz
tar -xzvf test_data/scif_test_data.tar.gz -C test_data/

The command line for running the tests is as follows:

# Assumes Python3.7+ is default Python version
python -m pytest -rpP

If pytest-cov is installed, it can be used to generate a code coverage report as part of running PyTest. The code coverage report shows how much of the code has been tested; it doesn't indicate how well that code has been tested. The modified PyTest command line including coverage is:

# Assumes Python3.7+ is default Python version
python -m pytest --cov=. -rpP 

Docker Testing

The Docker testing Workflow replicate the examples in this document to ensure they continue to work.