Semantic segmentation for precision agriculture on aerial images.

[vladtimug]

I am trying to solve a semantic segmentation task in the field of precision agriculture and I have some ortophoto drone images that have been acquired at the same height above the crops and in different vegetation stages of the crops. Right now I am trying to label the images in GIMP by marking all the pixels that cover crops in all the pictures and after trying to do this for the second day now I'm starting to think the images are not sufficiently detailed because it's hard for me to distinguish between pixels that are and aren't overlapping the crops in the field. (Either that or I just don't have enough labeling experience in which case I'm hoping someone here could share their piece of advice on this) I'm thinking if I have a hard time distinguishing between crop/no-crop it must be even more difficult for a segmentation model to do that. I've attached some images examples from my dataset and for reference an image from the Vainhingen ISPRS dataset zoomed at the same level to show a comparison between object detail level. Note the same zoom level between the 2 images and the level of detail of the yellow car in the Veinhingen dataset image. I would expect to see a similar detail level in my images to make the labeling effort worthwhile. What do you think?

[robmarkcole]

It is going to be a challenging task since the leaf is not a consistent colour, there are shadows, and leaves are irregular shapes. I suggest experimenting with the image contrast to see if you can enhance the difference between leaf and background. Possibly the images were acquired on an overcast day so naturally the contrast is low. I have been working on a similarly challenging task and it has taken several days of experimentation to get a process I am happy with. I actually implemented a custom annotation app at https://github.com/robmarkcole/streamlit-segmentation-app and this has enabled me to include all the required parameters to fine tune the presentation of images before annotation (not in the published code)

[vladtimug]

Funny, I had the same idea of using streamlit to build such a thing but I then I thought maybe it's better to invest the time and energy into annotating rather than building the annotation app itself. Thanks for the hints, I'll give it a shot. :)

[vladtimug]

@robmarkcole I realized that I will need to put together a preprocessing pipeline before even thinking about the semantic segmentation task, a way of "standardizing" all the data that comes in before getting it through the segmentation model. There are 2 questions I have at this point:

1. Do you have any recommendations of some literature I could investigate to understand how I should put together this preprocessing pipeline?
2. Do you think the annotation for the segmentation should occur on the raw or preprocessed image?

[robmarkcole]

I think you will see preprocessing discussed in papers, but it will be specific to that dataset. Are there any particularly unusual steps you anticipate? Generally it is a process of iteratively tweaking some python scripts

[robmarkcole]

You want your annotations to be as accurate as possible so I perform the preprocessing required to enable that